2024-01-28T11:44:23.056 INFO:root:teuthology version: 0.0.1.dev251+gc856e0f 2024-01-28T11:44:23.057 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T11:44:23.232 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9e85598c10>, signals=[15]) 2024-01-28T11:44:23.232 INFO:teuthology.lock.ops:Start node 'smithi005.front.sepia.ceph.com' reimaging 2024-01-28T11:44:23.232 INFO:teuthology.lock.ops:Updating [smithi005.front.sepia.ceph.com]: reset os type and version on server 2024-01-28T11:44:23.232 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-01-28T11:44:23.253 INFO:teuthology.lock.ops:Start node 'smithi160.front.sepia.ceph.com' reimaging 2024-01-28T11:44:23.254 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: reset os type and version on server 2024-01-28T11:44:23.254 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2024-01-28T11:44:23.278 INFO:teuthology.lock.ops:Start node 'smithi167.front.sepia.ceph.com' reimaging 2024-01-28T11:44:23.278 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: reset os type and version on server 2024-01-28T11:44:23.278 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-01-28T11:44:23.366 DEBUG:teuthology.provision.fog.smithi005:Requesting image smithi_rhel_8.6 (ID 5) 2024-01-28T11:44:23.379 DEBUG:teuthology.provision.fog.smithi160:Requesting image smithi_rhel_8.6 (ID 5) 2024-01-28T11:44:23.404 DEBUG:teuthology.provision.fog.smithi167:Requesting image smithi_rhel_8.6 (ID 5) 2024-01-28T11:44:23.433 INFO:teuthology.provision.fog.smithi005:Scheduling deploy of rhel 8.6 2024-01-28T11:44:23.436 INFO:teuthology.provision.fog.smithi160:Scheduling deploy of rhel 8.6 2024-01-28T11:44:23.451 INFO:teuthology.provision.fog.smithi167:Scheduling deploy of rhel 8.6 2024-01-28T11:44:23.710 INFO:teuthology.orchestra.console.smithi005:Power off 2024-01-28T11:44:23.711 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-28T11:44:23.735 DEBUG:teuthology.orchestra.console.smithi005:power off output: Chassis Power Control: Down/Off 2024-01-28T11:44:23.736 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:23.754 INFO:teuthology.orchestra.console.smithi160:Power off 2024-01-28T11:44:23.754 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-28T11:44:23.768 INFO:teuthology.orchestra.console.smithi167:Power off 2024-01-28T11:44:23.768 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-28T11:44:23.776 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2024-01-28T11:44:23.776 DEBUG:teuthology.orchestra.console.smithi160:power off output: Chassis Power Control: Down/Off 2024-01-28T11:44:23.776 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:23.794 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off 2024-01-28T11:44:23.794 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:23.804 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off 2024-01-28T11:44:23.829 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2024-01-28T11:44:23.905 INFO:teuthology.orchestra.console.smithi160:Power off completed 2024-01-28T11:44:24.005 INFO:teuthology.orchestra.console.smithi160:Power on 2024-01-28T11:44:24.006 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-28T11:44:24.026 DEBUG:teuthology.orchestra.console.smithi160:power on output: Chassis Power Control: Up/On 2024-01-28T11:44:24.027 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:24.047 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off 2024-01-28T11:44:27.780 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:27.854 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:27.956 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2024-01-28T11:44:27.963 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2024-01-28T11:44:28.049 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:28.157 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off 2024-01-28T11:44:31.960 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:31.967 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:32.070 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off 2024-01-28T11:44:32.076 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2024-01-28T11:44:32.158 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:32.171 INFO:teuthology.orchestra.console.smithi005:Power off completed 2024-01-28T11:44:32.268 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is on 2024-01-28T11:44:32.272 INFO:teuthology.orchestra.console.smithi005:Power on 2024-01-28T11:44:32.272 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-28T11:44:32.294 DEBUG:teuthology.orchestra.console.smithi005:power on output: Chassis Power Control: Up/On 2024-01-28T11:44:32.294 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:32.317 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off 2024-01-28T11:44:32.368 INFO:teuthology.orchestra.console.smithi160:Power on completed 2024-01-28T11:44:32.469 INFO:teuthology.provision.fog.smithi160:Waiting for deploy to finish 2024-01-28T11:44:36.080 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:36.192 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2024-01-28T11:44:36.293 INFO:teuthology.orchestra.console.smithi167:Power off completed 2024-01-28T11:44:36.318 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:36.393 INFO:teuthology.orchestra.console.smithi167:Power on 2024-01-28T11:44:36.394 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-28T11:44:36.418 DEBUG:teuthology.orchestra.console.smithi167:power on output: Chassis Power Control: Up/On 2024-01-28T11:44:36.418 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:36.431 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off 2024-01-28T11:44:36.442 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2024-01-28T11:44:40.433 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:40.443 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T11:44:40.542 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2024-01-28T11:44:40.551 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2024-01-28T11:44:40.644 INFO:teuthology.orchestra.console.smithi005:Power on completed 2024-01-28T11:44:40.652 INFO:teuthology.orchestra.console.smithi167:Power on completed 2024-01-28T11:44:40.745 INFO:teuthology.provision.fog.smithi005:Waiting for deploy to finish 2024-01-28T11:44:40.753 INFO:teuthology.provision.fog.smithi167:Waiting for deploy to finish 2024-01-28T11:47:11.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:47:18.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:47:26.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:47:29.980 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-01-28T11:47:35.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:47:36.892 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-01-28T11:47:39.036 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-01-28T11:47:42.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:47:45.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:47:45.088 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-01-28T11:47:45.948 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-01-28T11:47:48.096 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-01-28T11:47:51.092 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:47:51.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:47:54.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:47:54.172 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-01-28T11:47:55.008 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-01-28T11:47:57.180 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-01-28T11:48:00.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:48:01.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:48:03.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:48:03.228 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-01-28T11:48:04.064 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-01-28T11:48:09.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:48:10.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:48:12.316 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-01-28T11:48:13.116 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-01-28T11:48:18.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:48:19.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:48:21.376 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-01-28T11:48:22.176 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-01-28T11:48:27.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:48:28.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:48:46.524 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-01-28T11:48:52.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:48:55.584 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-01-28T11:49:01.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:49:03.188 WARNING:teuthology.provision.fog.smithi005:timed out 2024-01-28T11:49:04.672 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-01-28T11:49:09.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:49:09.439 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-28T11:49:09.440 DEBUG:teuthology.orchestra.run.smithi005:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-28T11:49:09.719 INFO:teuthology.provision.fog.smithi005:Node is ready 2024-01-28T11:49:09.719 DEBUG:teuthology.orchestra.run.smithi005:> hostname 2024-01-28T11:49:09.740 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005.front.sepia.ceph.com 2024-01-28T11:49:09.740 DEBUG:teuthology.orchestra.run.smithi005:> grep smithi005.front.sepia.ceph.com /etc/hosts 2024-01-28T11:49:09.802 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5 smithi005.front.sepia.ceph.com smithi005 2024-01-28T11:49:09.802 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname smithi005 2024-01-28T11:49:09.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i -e 's/smithi005.front.sepia.ceph.com/smithi005/g' /etc/hosts 2024-01-28T11:49:09.960 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i -e 's/172.21.15.5/172.21.15.5/g' /etc/hosts 2024-01-28T11:49:10.035 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i -e 's/smithi005.front.sepia.ceph.com/smithi005/g' /etc/hostname 2024-01-28T11:49:10.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname smithi005 2024-01-28T11:49:10.185 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-01-28T11:49:10.247 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="Red Hat Enterprise Linux" 2024-01-28T11:49:10.247 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="8.6 (Ootpa)" 2024-01-28T11:49:10.247 INFO:teuthology.orchestra.run.smithi005.stdout:ID="rhel" 2024-01-28T11:49:10.247 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="fedora" 2024-01-28T11:49:10.247 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="8.6" 2024-01-28T11:49:10.248 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el8" 2024-01-28T11:49:10.248 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-28T11:49:10.248 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-01-28T11:49:10.248 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-28T11:49:10.248 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://www.redhat.com/" 2024-01-28T11:49:10.248 INFO:teuthology.orchestra.run.smithi005.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-28T11:49:10.248 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-28T11:49:10.248 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-01-28T11:49:10.248 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-28T11:49:10.248 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-28T11:49:10.248 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-28T11:49:10.248 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-28T11:49:10.249 INFO:teuthology.provision.fog.smithi005:Deploy complete! 2024-01-28T11:49:10.250 DEBUG:teuthology.parallel:result is None 2024-01-28T11:49:10.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:49:11.627 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-28T11:49:11.627 DEBUG:teuthology.orchestra.run.smithi160:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-28T11:49:27.380 WARNING:teuthology.provision.fog.smithi167:timed out 2024-01-28T11:49:27.478 INFO:teuthology.provision.fog.smithi160:Node is ready 2024-01-28T11:49:27.478 DEBUG:teuthology.orchestra.run.smithi160:> hostname 2024-01-28T11:49:27.542 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160.front.sepia.ceph.com 2024-01-28T11:49:27.542 DEBUG:teuthology.orchestra.run.smithi160:> grep smithi160.front.sepia.ceph.com /etc/hosts 2024-01-28T11:49:27.604 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160 smithi160.front.sepia.ceph.com smithi160 2024-01-28T11:49:27.604 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname smithi160 2024-01-28T11:49:27.746 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/smithi160.front.sepia.ceph.com/smithi160/g' /etc/hosts 2024-01-28T11:49:27.776 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/172.21.15.160/172.21.15.160/g' /etc/hosts 2024-01-28T11:49:27.845 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/smithi160.front.sepia.ceph.com/smithi160/g' /etc/hostname 2024-01-28T11:49:27.915 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname smithi160 2024-01-28T11:49:27.984 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2024-01-28T11:49:28.041 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="Red Hat Enterprise Linux" 2024-01-28T11:49:28.042 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="8.6 (Ootpa)" 2024-01-28T11:49:28.042 INFO:teuthology.orchestra.run.smithi160.stdout:ID="rhel" 2024-01-28T11:49:28.042 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="fedora" 2024-01-28T11:49:28.042 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="8.6" 2024-01-28T11:49:28.042 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el8" 2024-01-28T11:49:28.042 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-28T11:49:28.042 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2024-01-28T11:49:28.042 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-28T11:49:28.042 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://www.redhat.com/" 2024-01-28T11:49:28.042 INFO:teuthology.orchestra.run.smithi160.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-28T11:49:28.042 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-28T11:49:28.042 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-01-28T11:49:28.043 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-28T11:49:28.043 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-28T11:49:28.043 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-28T11:49:28.043 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-28T11:49:28.043 INFO:teuthology.provision.fog.smithi160:Deploy complete! 2024-01-28T11:49:28.044 DEBUG:teuthology.parallel:result is None 2024-01-28T11:49:33.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:49:33.647 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-28T11:49:33.648 DEBUG:teuthology.orchestra.run.smithi167:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-28T11:49:33.888 INFO:teuthology.provision.fog.smithi167:Node is ready 2024-01-28T11:49:33.889 DEBUG:teuthology.orchestra.run.smithi167:> hostname 2024-01-28T11:49:33.907 INFO:teuthology.orchestra.run.smithi167.stdout:smithi167.front.sepia.ceph.com 2024-01-28T11:49:33.908 DEBUG:teuthology.orchestra.run.smithi167:> grep smithi167.front.sepia.ceph.com /etc/hosts 2024-01-28T11:49:33.965 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167 smithi167.front.sepia.ceph.com smithi167 2024-01-28T11:49:33.966 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167 2024-01-28T11:49:34.094 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hosts 2024-01-28T11:49:34.124 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/172.21.15.167/172.21.15.167/g' /etc/hosts 2024-01-28T11:49:34.193 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hostname 2024-01-28T11:49:34.262 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167 2024-01-28T11:49:34.329 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-01-28T11:49:34.387 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Red Hat Enterprise Linux" 2024-01-28T11:49:34.387 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8.6 (Ootpa)" 2024-01-28T11:49:34.387 INFO:teuthology.orchestra.run.smithi167.stdout:ID="rhel" 2024-01-28T11:49:34.387 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="fedora" 2024-01-28T11:49:34.387 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8.6" 2024-01-28T11:49:34.387 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8" 2024-01-28T11:49:34.387 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-28T11:49:34.388 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-01-28T11:49:34.388 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-28T11:49:34.388 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.redhat.com/" 2024-01-28T11:49:34.388 INFO:teuthology.orchestra.run.smithi167.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-28T11:49:34.388 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-28T11:49:34.388 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-01-28T11:49:34.388 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-28T11:49:34.388 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-28T11:49:34.388 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-28T11:49:34.388 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-28T11:49:34.388 INFO:teuthology.provision.fog.smithi167:Deploy complete! 2024-01-28T11:49:34.389 DEBUG:teuthology.parallel:result is None 2024-01-28T11:49:34.597 INFO:teuthology.lock.ops:Checking smithi005.front.sepia.ceph.com 2024-01-28T11:49:34.598 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-28T11:49:34.626 INFO:teuthology.lock.ops:Checking smithi160.front.sepia.ceph.com 2024-01-28T11:49:34.626 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-28T11:49:34.650 INFO:teuthology.lock.ops:Checking smithi167.front.sepia.ceph.com 2024-01-28T11:49:34.650 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-28T11:49:34.668 INFO:teuthology.lock.ops:Updating [smithi005.front.sepia.ceph.com]: set os type and version on server 2024-01-28T11:49:34.668 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-01-28T11:49:34.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:49:34.744 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-01-28T11:49:34.884 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-01-28T11:49:34.884 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-01-28T11:49:34.945 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-01-28T11:49:34.945 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-01-28T11:49:35.006 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="Red Hat Enterprise Linux" 2024-01-28T11:49:35.006 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="8.6 (Ootpa)" 2024-01-28T11:49:35.006 INFO:teuthology.orchestra.run.smithi005.stdout:ID="rhel" 2024-01-28T11:49:35.006 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="fedora" 2024-01-28T11:49:35.006 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="8.6" 2024-01-28T11:49:35.006 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el8" 2024-01-28T11:49:35.007 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-28T11:49:35.007 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-01-28T11:49:35.007 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-28T11:49:35.007 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://www.redhat.com/" 2024-01-28T11:49:35.007 INFO:teuthology.orchestra.run.smithi005.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-28T11:49:35.007 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-28T11:49:35.007 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-01-28T11:49:35.007 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-28T11:49:35.007 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-28T11:49:35.007 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-28T11:49:35.007 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-28T11:49:35.008 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-01-28T11:49:35.034 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: set os type and version on server 2024-01-28T11:49:35.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-01-28T11:49:35.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:49:35.105 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-01-28T11:49:35.133 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-01-28T11:49:35.133 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2024-01-28T11:49:35.192 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2024-01-28T11:49:35.192 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2024-01-28T11:49:35.253 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="Red Hat Enterprise Linux" 2024-01-28T11:49:35.253 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="8.6 (Ootpa)" 2024-01-28T11:49:35.254 INFO:teuthology.orchestra.run.smithi160.stdout:ID="rhel" 2024-01-28T11:49:35.254 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="fedora" 2024-01-28T11:49:35.254 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="8.6" 2024-01-28T11:49:35.254 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el8" 2024-01-28T11:49:35.254 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-28T11:49:35.254 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2024-01-28T11:49:35.254 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-28T11:49:35.254 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://www.redhat.com/" 2024-01-28T11:49:35.254 INFO:teuthology.orchestra.run.smithi160.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-28T11:49:35.254 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-28T11:49:35.254 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-01-28T11:49:35.254 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-28T11:49:35.254 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-28T11:49:35.254 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-28T11:49:35.255 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-28T11:49:35.255 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2024-01-28T11:49:35.275 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: set os type and version on server 2024-01-28T11:49:35.275 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-01-28T11:49:35.276 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T11:49:35.350 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-01-28T11:49:35.418 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-01-28T11:49:35.418 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-01-28T11:49:35.477 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-01-28T11:49:35.477 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-01-28T11:49:35.535 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Red Hat Enterprise Linux" 2024-01-28T11:49:35.536 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8.6 (Ootpa)" 2024-01-28T11:49:35.536 INFO:teuthology.orchestra.run.smithi167.stdout:ID="rhel" 2024-01-28T11:49:35.536 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="fedora" 2024-01-28T11:49:35.536 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8.6" 2024-01-28T11:49:35.536 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8" 2024-01-28T11:49:35.536 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-28T11:49:35.536 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-01-28T11:49:35.536 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-28T11:49:35.536 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.redhat.com/" 2024-01-28T11:49:35.536 INFO:teuthology.orchestra.run.smithi167.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-28T11:49:35.536 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-28T11:49:35.537 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-01-28T11:49:35.537 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-28T11:49:35.537 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-28T11:49:35.537 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-28T11:49:35.537 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-28T11:49:35.537 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-01-28T11:49:35.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T11:49:35.654 INFO:teuthology.dispatcher.supervisor:Running job 7535757 2024-01-28T11:49:35.654 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/bin/teuthology -v --owner scheduled_khiremat@teuthology --archive /home/teuthworker/archive/khiremat-2024-01-28_10:21:48-fs:workload-wip-khiremat-49945-referent-inode-3-distro-default-smithi/7535757 --name khiremat-2024-01-28_10:21:48-fs:workload-wip-khiremat-49945-referent-inode-3-distro-default-smithi --description fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsstress}} -- /home/teuthworker/archive/khiremat-2024-01-28_10:21:48-fs:workload-wip-khiremat-49945-referent-inode-3-distro-default-smithi/7535757/orig.config.yaml 2024-01-28T11:49:35.657 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/khiremat-2024-01-28_10:21:48-fs:workload-wip-khiremat-49945-referent-inode-3-distro-default-smithi/7535757 2024-01-28T11:49:35.657 INFO:teuthology.dispatcher.supervisor:Job PID: 166345 2024-01-28T11:49:35.657 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-01-28T11:51:35.758 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T11:53:35.928 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T11:55:36.108 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T11:57:36.348 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T11:59:36.524 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:01:36.713 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:03:36.876 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:05:37.060 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:07:37.233 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:09:37.405 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:11:37.569 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:13:37.660 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:15:37.836 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:17:37.977 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:19:38.150 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:21:38.324 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:23:38.486 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:25:38.659 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:27:38.853 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:29:39.014 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:31:39.148 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:33:39.332 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:35:39.513 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:37:39.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:39:39.844 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:41:39.963 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:43:40.136 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:45:40.256 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:47:40.397 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:49:40.564 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:51:40.741 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:53:41.147 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:55:41.308 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:57:41.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:59:41.583 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:01:41.661 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:03:41.827 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:05:41.993 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:07:42.147 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:09:42.316 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:11:42.484 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:13:42.649 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:15:42.811 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:17:42.964 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:19:43.058 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:21:43.132 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:23:43.288 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:25:43.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:27:43.646 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:29:43.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:31:43.976 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:33:44.150 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:35:44.324 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:37:44.484 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:39:44.636 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:41:44.810 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:43:44.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:45:45.162 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:47:45.332 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:49:45.499 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:51:45.628 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:53:45.796 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:55:45.927 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:57:46.088 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:59:46.225 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:01:46.389 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:03:46.556 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:05:46.716 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:07:46.882 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:09:47.056 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:11:47.222 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:13:47.372 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:15:47.544 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:17:47.710 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:19:47.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:21:48.040 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:23:48.162 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:25:48.324 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:27:48.478 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:29:48.636 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:31:48.815 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:33:48.980 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:35:49.134 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:37:49.297 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:39:49.460 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:41:49.622 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:43:49.782 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:45:49.945 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:47:50.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:49:50.241 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:51:50.412 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:53:50.564 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:55:50.728 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:57:50.888 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T14:59:51.054 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:01:51.194 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:03:51.360 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:05:51.520 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:07:51.691 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:09:51.844 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:11:52.006 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:13:52.164 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:15:52.332 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:17:52.488 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:19:52.638 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:21:52.807 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:23:52.985 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:25:53.115 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:27:53.272 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:29:53.438 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:31:53.604 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:33:53.749 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:35:53.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:37:54.080 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:39:54.245 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:41:54.384 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:43:54.556 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:45:54.728 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:47:54.884 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:49:55.045 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:51:55.196 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:53:55.365 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:55:55.533 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:57:55.680 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T15:59:55.834 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:01:56.000 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:03:56.176 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:05:56.348 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:07:56.525 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:09:56.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:11:56.860 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:13:57.010 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:15:57.176 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:17:57.333 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:19:57.489 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:21:57.648 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:23:57.807 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:25:57.964 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:27:58.118 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:29:58.288 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:31:58.452 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:33:58.621 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:35:58.784 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:37:58.956 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:39:59.120 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:41:59.275 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:43:59.444 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:45:59.609 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:47:59.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:49:59.884 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:52:00.048 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:54:00.177 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:56:00.336 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T16:58:00.755 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:00:00.938 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:02:01.108 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:04:01.308 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:06:01.452 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:08:01.620 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:10:01.781 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:12:01.948 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:14:02.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:16:02.277 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:18:02.440 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:20:02.499 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:22:02.661 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:24:02.816 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:26:02.976 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:28:03.151 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:30:03.318 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:32:03.492 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:34:03.640 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:36:03.743 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:38:03.912 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:40:04.089 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:42:04.258 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:44:04.427 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:46:04.592 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:48:04.758 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:50:04.929 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:52:05.095 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:54:05.172 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:56:05.342 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T17:58:05.504 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:00:05.693 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:02:05.855 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:04:06.008 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:06:06.176 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:08:06.336 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:10:06.505 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:12:06.660 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:14:06.824 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:16:06.950 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:18:07.116 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:20:07.284 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:22:07.444 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:24:07.605 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:26:07.759 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:28:07.895 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:30:08.050 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:32:08.205 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:34:08.373 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:36:08.530 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:38:08.677 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:40:08.856 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:42:08.979 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:44:09.148 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:46:09.316 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:48:09.493 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:50:09.664 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:52:09.820 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:54:09.994 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:56:10.149 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T18:58:10.308 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:00:10.472 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:02:10.637 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:04:10.804 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:06:10.977 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:08:11.145 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:10:11.306 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:12:11.478 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:14:11.632 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:16:11.799 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:18:11.961 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:20:12.125 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:22:12.293 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:24:12.460 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:26:12.634 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:28:12.800 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:30:12.933 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:32:13.094 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:34:13.261 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:36:13.421 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:38:13.558 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:40:13.740 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:42:13.899 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:44:14.061 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:46:14.209 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:48:14.341 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:50:14.510 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:52:14.671 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:54:14.830 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:56:14.997 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T19:58:15.164 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:00:15.316 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:02:15.484 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:04:15.640 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:06:15.785 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:08:15.952 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:10:16.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:12:17.100 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:14:17.268 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:16:17.435 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:18:17.562 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:20:17.744 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:22:17.876 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:24:18.019 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:26:18.110 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:28:18.281 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:30:18.444 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:32:18.610 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:34:18.774 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:36:18.944 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:38:19.116 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:40:19.290 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:42:19.461 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:44:19.626 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:46:19.731 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:48:19.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:50:20.085 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:52:20.252 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:54:20.410 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:56:20.580 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T20:58:20.745 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:00:20.920 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:02:21.084 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:04:21.244 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:06:21.412 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:08:21.588 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:10:21.768 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:12:21.927 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:14:22.102 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:16:22.273 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:18:22.448 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:20:22.544 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:22:22.694 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:24:22.845 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:26:23.022 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:28:23.163 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:30:23.334 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:32:23.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:34:23.634 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:36:23.801 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:38:23.972 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:40:24.139 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:42:24.307 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:44:24.480 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:46:24.618 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:48:24.767 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:50:24.931 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:52:25.084 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:54:25.261 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:56:25.421 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T21:58:25.569 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:00:25.736 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:02:25.906 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:04:26.082 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:06:26.197 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:08:26.341 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:10:26.497 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:12:26.659 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:14:26.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:16:26.942 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:18:27.066 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:20:27.236 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:22:27.352 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:24:27.523 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:26:27.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:28:27.857 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:30:28.028 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:32:28.191 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:34:28.349 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:36:28.512 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:38:28.672 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:40:28.831 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:42:29.002 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:44:29.124 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:46:29.235 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:48:29.400 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:50:29.572 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:52:29.737 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:54:29.898 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:56:30.061 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T22:58:30.231 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:00:30.391 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:02:30.531 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:04:30.700 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:06:30.872 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:08:31.109 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:10:31.272 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:12:31.440 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:14:31.595 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:16:31.765 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:18:31.929 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:20:32.109 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:22:32.276 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:24:32.443 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:26:32.608 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:28:32.776 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:30:32.948 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:32:33.111 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:34:33.280 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:36:33.668 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:38:33.832 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:40:33.986 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:42:34.147 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:44:34.253 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:46:34.399 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:48:34.520 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:50:34.682 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2024-01-28T23:50:34.828 INFO:teuthology.kill:Killing Pids: {166345} 2024-01-28T23:50:35.805 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-01-28T23:50:35.805 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-01-28T23:50:35.805 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-01-28T23:50:35.806 INFO:teuthology.misc:Compressing logs... 2024-01-28T23:50:35.806 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-01-28T23:50:35.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T23:50:35.887 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-01-28T23:50:35.964 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-01-28T23:50:35.964 DEBUG:teuthology.orchestra.run.smithi005:> sudo find /var/lib/ceph/crash -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-28T23:50:36.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-01-28T23:50:36.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T23:50:36.085 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-01-28T23:50:36.157 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-01-28T23:50:36.157 DEBUG:teuthology.orchestra.run.smithi160:> sudo find /var/lib/ceph/crash -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-28T23:50:36.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-01-28T23:50:36.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T23:50:36.278 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-01-28T23:50:36.353 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-01-28T23:50:36.353 DEBUG:teuthology.orchestra.run.smithi167:> sudo find /var/lib/ceph/crash -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-28T23:50:36.455 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/crash to /home/teuthworker/archive/khiremat-2024-01-28_10:21:48-fs:workload-wip-khiremat-49945-referent-inode-3-distro-default-smithi/7535757/remote/smithi005/crash 2024-01-28T23:50:36.456 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-28T23:50:36.493 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/crash to /home/teuthworker/archive/khiremat-2024-01-28_10:21:48-fs:workload-wip-khiremat-49945-referent-inode-3-distro-default-smithi/7535757/remote/smithi160/crash 2024-01-28T23:50:36.494 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-28T23:50:36.538 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/crash to /home/teuthworker/archive/khiremat-2024-01-28_10:21:48-fs:workload-wip-khiremat-49945-referent-inode-3-distro-default-smithi/7535757/remote/smithi167/crash 2024-01-28T23:50:36.539 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-28T23:50:36.580 INFO:teuthology.misc:Compressing logs... 2024-01-28T23:50:36.580 DEBUG:teuthology.orchestra.run.smithi005:> sudo find /home/ubuntu/cephtest/archive -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-28T23:50:36.583 DEBUG:teuthology.orchestra.run.smithi160:> sudo find /home/ubuntu/cephtest/archive -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-28T23:50:36.585 DEBUG:teuthology.orchestra.run.smithi167:> sudo find /home/ubuntu/cephtest/archive -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-28T23:50:36.612 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-28T23:50:36.612 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-28T23:50:36.613 INFO:teuthology.orchestra.run.smithi160.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 82.1% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-01-28T23:50:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-28T23:50:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-28T23:50:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 87.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-01-28T23:50:36.651 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-28T23:50:36.651 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-28T23:50:36.652 INFO:teuthology.orchestra.run.smithi167.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 82.1% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-01-28T23:50:36.677 INFO:teuthology.orchestra.run.smithi160.stderr: 89.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-01-28T23:50:36.694 INFO:teuthology.orchestra.run.smithi005.stderr: 89.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-01-28T23:50:36.714 INFO:teuthology.orchestra.run.smithi167.stderr: 89.7% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-01-28T23:50:36.716 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/khiremat-2024-01-28_10:21:48-fs:workload-wip-khiremat-49945-referent-inode-3-distro-default-smithi/7535757/remote/smithi005/ 2024-01-28T23:50:36.716 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-28T23:50:36.783 DEBUG:teuthology.misc:Transferring archived files from smithi160:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/khiremat-2024-01-28_10:21:48-fs:workload-wip-khiremat-49945-referent-inode-3-distro-default-smithi/7535757/remote/smithi160/ 2024-01-28T23:50:36.784 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-28T23:50:36.822 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/khiremat-2024-01-28_10:21:48-fs:workload-wip-khiremat-49945-referent-inode-3-distro-default-smithi/7535757/remote/smithi167/ 2024-01-28T23:50:36.823 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-28T23:50:36.862 INFO:teuthology.misc:Compressing logs... 2024-01-28T23:50:36.862 DEBUG:teuthology.orchestra.run.smithi005:> sudo find /var/log/ceph -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-28T23:50:36.864 DEBUG:teuthology.orchestra.run.smithi160:> sudo find /var/log/ceph -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-28T23:50:36.867 DEBUG:teuthology.orchestra.run.smithi167:> sudo find /var/log/ceph -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-28T23:50:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-01-28T23:50:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.7.log 2024-01-28T23:50:36.893 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.1.log 2024-01-28T23:50:36.893 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.h.log 2024-01-28T23:50:36.893 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.7.log: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-volume.log 2024-01-28T23:50:36.893 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.k.log 2024-01-28T23:50:36.894 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.h.log: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.e.log 2024-01-28T23:50:36.897 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-01-28T23:50:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mon.a.log 2024-01-28T23:50:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.g.log 2024-01-28T23:50:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.j.log 2024-01-28T23:50:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.a.log /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.g.log: 2024-01-28T23:50:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.9.log 2024-01-28T23:50:36.904 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.k.log: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.e.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-01-28T23:50:36.904 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mon.b.log 2024-01-28T23:50:36.906 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.b.log 2024-01-28T23:50:36.906 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.a.log: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.3.log 2024-01-28T23:50:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.3.log: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-01-28T23:50:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-volume.log 2024-01-28T23:50:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.0.log 2024-01-28T23:50:36.915 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mgr.y.log 2024-01-28T23:50:36.917 INFO:teuthology.orchestra.run.smithi160.stderr: 91.5% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-volume.log.gz 2024-01-28T23:50:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-volume.log: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.d.log 2024-01-28T23:50:36.928 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-01-28T23:50:36.929 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.5.log 2024-01-28T23:50:36.929 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.i.log 2024-01-28T23:50:36.929 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.c.log 2024-01-28T23:50:36.929 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.10.log 2024-01-28T23:50:36.939 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.5.log: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.i.log: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.c.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-01-28T23:50:36.939 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.f.log 2024-01-28T23:50:36.939 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mgr.z.log 2024-01-28T23:50:36.939 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.11.log 2024-01-28T23:50:36.939 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.2.log 2024-01-28T23:50:36.939 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.11.log: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-volume.log 2024-01-28T23:50:36.948 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.4.log 2024-01-28T23:50:36.948 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.6.log 2024-01-28T23:50:36.950 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.l.log 2024-01-28T23:50:36.961 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mon.c.log 2024-01-28T23:50:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.6.log: 91.6% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-volume.log.gz 2024-01-28T23:50:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mgr.x.log 2024-01-28T23:50:36.972 INFO:teuthology.orchestra.run.smithi167.stderr: 91.5% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-volume.log.gz 2024-01-28T23:50:36.983 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.8.log 2024-01-28T23:50:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-01-28T23:50:37.045 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.4.log: 94.5% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mgr.y.log.gz 2024-01-28T23:50:37.058 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.8.log: 94.6% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mgr.z.log.gz 2024-01-28T23:50:39.766 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.f.log: file size changed while zipping 2024-01-28T23:50:39.766 INFO:teuthology.orchestra.run.smithi167.stderr: 95.0% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.f.log.gz 2024-01-28T23:50:39.849 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.c.log: file size changed while zipping 2024-01-28T23:50:39.849 INFO:teuthology.orchestra.run.smithi167.stderr: 95.1% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.c.log.gz 2024-01-28T23:50:46.311 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.h.log: file size changed while zipping 2024-01-28T23:50:46.311 INFO:teuthology.orchestra.run.smithi160.stderr: 95.0% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.h.log.gz 2024-01-28T23:50:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.log: gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mgr.x.log: file size changed while zipping 2024-01-28T23:50:47.271 INFO:teuthology.orchestra.run.smithi005.stderr: 92.8% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mgr.x.log.gz 2024-01-28T23:50:47.950 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mon.c.log: file size changed while zipping 2024-01-28T23:50:47.951 INFO:teuthology.orchestra.run.smithi167.stderr: 92.1% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mon.c.log.gz 2024-01-28T23:50:48.110 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.i.log: file size changed while zipping 2024-01-28T23:50:48.110 INFO:teuthology.orchestra.run.smithi167.stderr: 92.2% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.i.log.gz 2024-01-28T23:50:49.443 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mon.b.log: file size changed while zipping 2024-01-28T23:50:49.443 INFO:teuthology.orchestra.run.smithi160.stderr: 92.3% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mon.b.log.gz 2024-01-28T23:50:51.308 INFO:teuthology.orchestra.run.smithi167.stderr: 95.4% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.l.log.gz 2024-01-28T23:50:53.449 INFO:teuthology.orchestra.run.smithi160.stderr: 95.4% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.k.log.gz 2024-01-28T23:50:54.112 INFO:teuthology.orchestra.run.smithi160.stderr: 95.4% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.e.log.gz 2024-01-28T23:50:54.933 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.2.log: file size changed while zipping 2024-01-28T23:50:54.933 INFO:teuthology.orchestra.run.smithi167.stderr: 92.5% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.2.log.gz 2024-01-28T23:50:55.166 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.11.log: file size changed while zipping 2024-01-28T23:50:55.166 INFO:teuthology.orchestra.run.smithi167.stderr: 92.7% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.11.log.gz 2024-01-28T23:50:56.361 INFO:teuthology.orchestra.run.smithi005.stderr: 95.4% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.a.log.gz 2024-01-28T23:50:57.211 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.4.log: file size changed while zipping 2024-01-28T23:50:57.211 INFO:teuthology.orchestra.run.smithi160.stderr: 92.5% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.4.log.gz 2024-01-28T23:50:57.250 INFO:teuthology.orchestra.run.smithi005.stderr: 95.4% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.j.log.gz 2024-01-28T23:50:59.332 INFO:teuthology.orchestra.run.smithi005.stderr: 95.4% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.g.log.gz 2024-01-28T23:50:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.1.log: file size changed while zipping 2024-01-28T23:50:59.636 INFO:teuthology.orchestra.run.smithi160.stderr: 92.5% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.1.log.gz 2024-01-28T23:50:59.701 INFO:teuthology.orchestra.run.smithi005.stderr: 95.4% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.d.log.gz 2024-01-28T23:51:00.435 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.8.log: file size changed while zipping 2024-01-28T23:51:00.435 INFO:teuthology.orchestra.run.smithi167.stderr: 92.4% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.8.log.gz 2024-01-28T23:51:02.815 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.10.log: file size changed while zipping 2024-01-28T23:51:02.816 INFO:teuthology.orchestra.run.smithi160.stderr: 92.4% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.10.log.gz 2024-01-28T23:51:03.159 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.9.log: file size changed while zipping 2024-01-28T23:51:03.159 INFO:teuthology.orchestra.run.smithi005.stderr: 92.3% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.9.log.gz 2024-01-28T23:51:03.220 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.5.log: file size changed while zipping 2024-01-28T23:51:03.220 INFO:teuthology.orchestra.run.smithi167.stderr: 92.6% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.5.log.gz 2024-01-28T23:51:03.305 INFO:teuthology.orchestra.run.smithi005.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-01-28T23:51:03.316 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.7.log: file size changed while zipping 2024-01-28T23:51:03.316 INFO:teuthology.orchestra.run.smithi160.stderr: 92.6% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.7.log.gz 2024-01-28T23:51:04.901 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.3.log: file size changed while zipping 2024-01-28T23:51:04.901 INFO:teuthology.orchestra.run.smithi005.stderr: 92.7% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.3.log.gz 2024-01-28T23:51:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.0.log: file size changed while zipping 2024-01-28T23:51:05.310 INFO:teuthology.orchestra.run.smithi005.stderr: 92.6% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.0.log.gz 2024-01-28T23:51:11.712 INFO:teuthology.orchestra.run.smithi160.stderr: 94.1% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mds.b.log.gz 2024-01-28T23:51:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.6.log: file size changed while zipping 2024-01-28T23:51:11.992 INFO:teuthology.orchestra.run.smithi005.stderr: 92.5% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-osd.6.log.gz 2024-01-28T23:51:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mon.a.log: file size changed while zipping 2024-01-28T23:51:12.045 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/4e6047b8-bdd5-11ee-95b2-87774f69a715/ceph-mon.a.log.gz 2024-01-28T23:51:12.047 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/khiremat-2024-01-28_10:21:48-fs:workload-wip-khiremat-49945-referent-inode-3-distro-default-smithi/7535757/remote/smithi005/log 2024-01-28T23:51:12.048 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-28T23:51:32.933 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/log/ceph to /home/teuthworker/archive/khiremat-2024-01-28_10:21:48-fs:workload-wip-khiremat-49945-referent-inode-3-distro-default-smithi/7535757/remote/smithi160/log 2024-01-28T23:51:32.935 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-28T23:51:52.483 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/khiremat-2024-01-28_10:21:48-fs:workload-wip-khiremat-49945-referent-inode-3-distro-default-smithi/7535757/remote/smithi167/log 2024-01-28T23:51:52.516 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-28T23:52:08.132 INFO:teuthology.kill:No teuthology processes running 2024-01-28T23:52:09.134 INFO:teuthology.kill:Nuking machines: ['smithi005', 'smithi160', 'smithi167'] 2024-01-28T23:52:09.963 INFO:teuthology.kill:2024-01-28 23:52:09,963.963 INFO:teuthology.nuke:targets: 2024-01-28T23:52:09.964 INFO:teuthology.kill: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAtQZzRCltZl4xJR3WsLckiOyPUos9D599lcZOMOzhQRp3qz0Y2AopKUfqF0pHc94/QUod4CTln0hXj1WfmiB+8= 2024-01-28T23:52:09.964 INFO:teuthology.kill: smithi160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNimzJvTcUHwK7cbEJ2NKjQos/s0YJG1ykYrxeXxzBJkZnwRl6X1BbQccefEFBGcjva/fmzuaq9ZW8QZvU6LGTA= 2024-01-28T23:52:09.964 INFO:teuthology.kill: smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBUmmQMfC2C3d5h1EUouBCANqSSod3xyKSTbqKllMoFm0biQg2XJAHbm8mP1tlgzf6FozsxIfNIJsxHfy9VcAak= 2024-01-28T23:52:09.964 INFO:teuthology.kill:2024-01-28 23:52:09,963.963 INFO:teuthology.nuke:Checking targets against current locks 2024-01-28T23:52:10.020 INFO:teuthology.kill:2024-01-28 23:52:10,020.020 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-28T23:52:10.041 INFO:teuthology.kill:2024-01-28 23:52:10,041.041 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-28T23:52:10.053 INFO:teuthology.kill:2024-01-28 23:52:10,053.053 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-28T23:52:10.085 INFO:teuthology.kill:2024-01-28 23:52:10,084.084 INFO:teuthology.orchestra.console.smithi005:Power off 2024-01-28T23:52:10.138 INFO:teuthology.kill:2024-01-28 23:52:10,138.138 INFO:teuthology.orchestra.console.smithi160:Power off 2024-01-28T23:52:10.151 INFO:teuthology.kill:2024-01-28 23:52:10,151.151 INFO:teuthology.orchestra.console.smithi167:Power off 2024-01-28T23:52:18.510 INFO:teuthology.kill:2024-01-28 23:52:18,509.509 INFO:teuthology.orchestra.console.smithi167:Power off completed 2024-01-28T23:52:18.658 INFO:teuthology.kill:2024-01-28 23:52:18,657.657 INFO:teuthology.lock.ops:unlocked: smithi167.front.sepia.ceph.com 2024-01-28T23:52:22.602 INFO:teuthology.kill:2024-01-28 23:52:22,602.602 INFO:teuthology.orchestra.console.smithi005:Power off completed 2024-01-28T23:52:22.613 INFO:teuthology.kill:2024-01-28 23:52:22,613.613 INFO:teuthology.orchestra.console.smithi160:Power off completed 2024-01-28T23:52:22.739 INFO:teuthology.kill:2024-01-28 23:52:22,739.739 INFO:teuthology.lock.ops:unlocked: smithi005.front.sepia.ceph.com 2024-01-28T23:52:22.757 INFO:teuthology.kill:2024-01-28 23:52:22,757.757 INFO:teuthology.lock.ops:unlocked: smithi160.front.sepia.ceph.com 2024-01-28T23:52:22.838 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:54:23.005 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T23:54:23.082 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15