2023-05-21T13:58:28.318 INFO:root:teuthology version: 0.0.1.dev101+gc6e0984 2023-05-21T13:58:28.515 INFO:teuthology.lock.ops:Start node 'smithi012.front.sepia.ceph.com' reimaging 2023-05-21T13:58:28.517 INFO:teuthology.lock.ops:Updating [smithi012.front.sepia.ceph.com]: reset os type and version on server 2023-05-21T13:58:28.517 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2023-05-21T13:58:28.551 INFO:teuthology.lock.ops:Node 'smithi012.front.sepia.ceph.com' reimaging is complete 2023-05-21T13:58:28.552 INFO:teuthology.lock.ops:Start node 'smithi124.front.sepia.ceph.com' reimaging 2023-05-21T13:58:28.552 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: reset os type and version on server 2023-05-21T13:58:28.553 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2023-05-21T13:58:28.576 INFO:teuthology.lock.ops:Node 'smithi124.front.sepia.ceph.com' reimaging is complete 2023-05-21T13:58:28.741 INFO:teuthology.provision.fog.smithi012:Scheduling deploy of ubuntu 22.04 2023-05-21T13:58:28.750 INFO:teuthology.provision.fog.smithi124:Scheduling deploy of ubuntu 22.04 2023-05-21T13:58:29.022 INFO:teuthology.orchestra.console:Power off smithi012 2023-05-21T13:58:29.040 INFO:teuthology.orchestra.console:Power off smithi124 2023-05-21T13:58:37.402 INFO:teuthology.orchestra.console:Power off for smithi012 completed 2023-05-21T13:58:37.408 INFO:teuthology.orchestra.console:Power off for smithi124 completed 2023-05-21T13:58:37.503 INFO:teuthology.orchestra.console:Power on smithi012 2023-05-21T13:58:37.514 INFO:teuthology.orchestra.console:Power on smithi124 2023-05-21T13:58:45.877 INFO:teuthology.orchestra.console:Power on for smithi124 completed 2023-05-21T13:58:45.896 INFO:teuthology.orchestra.console:Power on for smithi012 completed 2023-05-21T13:58:45.979 INFO:teuthology.provision.fog.smithi124:Waiting for deploy to finish 2023-05-21T13:58:45.997 INFO:teuthology.provision.fog.smithi012:Waiting for deploy to finish 2023-05-21T14:01:50.064 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-05-21T14:01:59.156 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-05-21T14:02:05.104 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-05-21T14:02:08.208 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-05-21T14:02:14.161 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-05-21T14:02:17.264 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-05-21T14:02:23.216 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-05-21T14:02:26.324 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-05-21T14:02:32.276 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-05-21T14:02:41.360 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-05-21T14:02:50.416 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-05-21T14:03:32.364 WARNING:teuthology.provision.fog:timed out 2023-05-21T14:03:38.375 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-05-21T14:03:44.671 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-05-21T14:03:45.111 INFO:teuthology.provision.fog.smithi012:Node is ready 2023-05-21T14:03:45.157 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012.front.sepia.ceph.com 2023-05-21T14:03:45.205 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12 smithi012.front.sepia.ceph.com smithi012 2023-05-21T14:03:45.508 INFO:teuthology.provision.fog.smithi012:Deploy complete! 2023-05-21T14:03:56.424 WARNING:teuthology.provision.fog:timed out 2023-05-21T14:04:02.616 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-05-21T14:04:03.117 INFO:teuthology.provision.fog.smithi124:Node is ready 2023-05-21T14:04:03.164 INFO:teuthology.orchestra.run.smithi124.stdout:smithi124.front.sepia.ceph.com 2023-05-21T14:04:03.212 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.15.124 smithi124.front.sepia.ceph.com smithi124 2023-05-21T14:04:03.526 INFO:teuthology.provision.fog.smithi124:Deploy complete! 2023-05-21T14:04:03.713 INFO:teuthology.lock.ops:Checking smithi012.front.sepia.ceph.com 2023-05-21T14:04:03.714 INFO:teuthology.lock.ops:New key found. Updating... 2023-05-21T14:04:03.742 INFO:teuthology.lock.ops:Checking smithi124.front.sepia.ceph.com 2023-05-21T14:04:03.743 INFO:teuthology.lock.ops:New key found. Updating... 2023-05-21T14:04:03.769 INFO:teuthology.lock.ops:Updating [smithi012.front.sepia.ceph.com]: set os type and version on server 2023-05-21T14:04:03.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-05-21T14:04:04.004 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2023-05-21T14:04:04.053 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-05-21T14:04:04.053 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Ubuntu" 2023-05-21T14:04:04.054 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="22.04" 2023-05-21T14:04:04.054 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-05-21T14:04:04.054 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_CODENAME=jammy 2023-05-21T14:04:04.054 INFO:teuthology.orchestra.run.smithi012.stdout:ID=ubuntu 2023-05-21T14:04:04.054 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE=debian 2023-05-21T14:04:04.054 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.ubuntu.com/" 2023-05-21T14:04:04.054 INFO:teuthology.orchestra.run.smithi012.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-05-21T14:04:04.055 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-05-21T14:04:04.055 INFO:teuthology.orchestra.run.smithi012.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-05-21T14:04:04.055 INFO:teuthology.orchestra.run.smithi012.stdout:UBUNTU_CODENAME=jammy 2023-05-21T14:04:04.056 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2023-05-21T14:04:04.076 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: set os type and version on server 2023-05-21T14:04:04.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-05-21T14:04:04.290 INFO:teuthology.orchestra.run.smithi124.stdout:x86_64 2023-05-21T14:04:04.334 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-05-21T14:04:04.335 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="Ubuntu" 2023-05-21T14:04:04.335 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="22.04" 2023-05-21T14:04:04.335 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-05-21T14:04:04.335 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_CODENAME=jammy 2023-05-21T14:04:04.335 INFO:teuthology.orchestra.run.smithi124.stdout:ID=ubuntu 2023-05-21T14:04:04.336 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE=debian 2023-05-21T14:04:04.336 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://www.ubuntu.com/" 2023-05-21T14:04:04.336 INFO:teuthology.orchestra.run.smithi124.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-05-21T14:04:04.336 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-05-21T14:04:04.336 INFO:teuthology.orchestra.run.smithi124.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-05-21T14:04:04.336 INFO:teuthology.orchestra.run.smithi124.stdout:UBUNTU_CODENAME=jammy 2023-05-21T14:04:04.337 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2023-05-21T14:04:04.475 INFO:teuthology.dispatcher.supervisor:Running job 7281668 2023-05-21T14:04:04.476 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2023-05-21_12:08:41-fs-wip-pdonnell-testing-20230519.164114-distro-default-smithi/7281668 --name pdonnell-2023-05-21_12:08:41-fs-wip-pdonnell-testing-20230519.164114-distro-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} -- /home/teuthworker/archive/pdonnell-2023-05-21_12:08:41-fs-wip-pdonnell-testing-20230519.164114-distro-default-smithi/7281668/orig.config.yaml 2023-05-21T14:04:04.480 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2023-05-21_12:08:41-fs-wip-pdonnell-testing-20230519.164114-distro-default-smithi/7281668 2023-05-21T14:04:04.481 INFO:teuthology.dispatcher.supervisor:Job PID: 28712 2023-05-21T14:04:04.481 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-05-22T02:05:13.367 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2023-05-22T02:05:16.891 INFO:teuthology.kill:Killing Pids: {28712} 2023-05-22T02:05:22.684 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2023-05-22T02:05:22.702 INFO:teuthology.task.internal:roles: ubuntu@smithi124.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2023-05-22T02:05:22.732 INFO:teuthology.misc:Compressing logs... 2023-05-22T02:05:22.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-05-22T02:05:24.917 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-05-22T02:05:24.952 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-22T02:05:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-22T02:05:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-05-22T02:05:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 79.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-05-22T02:05:25.129 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-22T02:05:25.129 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-22T02:05:25.130 INFO:teuthology.orchestra.run.smithi124.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 77.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-05-22T02:05:25.158 INFO:teuthology.orchestra.run.smithi124.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 87.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-05-22T02:05:26.091 INFO:teuthology.kill:No teuthology processes running 2023-05-22T02:05:27.061 INFO:teuthology.kill:No locked machines. Not nuking anything 2023-05-22T02:07:27.373 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2023-05-22T02:07:27.995 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi012 but it was locked by another job: /home/teuthworker/archive/sjust-2023-05-21_21:38:43-crimson-rados-wip-matanb-crimson-testing-21.5-reef-distro-default-smithi/7281968 2023-05-22T02:07:27.996 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi124 but it was locked by another job: /home/teuthworker/archive/adking-2023-05-21_23:26:59-orch:cephadm-wip-adk3-testing-2023-05-21-1607-quincy-distro-default-smithi/7282012