2023-12-16T00:49:27.359 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e
2023-12-16T00:49:27.359 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T00:49:27.499 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fbf967704f0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fbf976b6940>, signals=[15])
2023-12-16T00:49:27.500 INFO:teuthology.lock.ops:Start node 'smithi005.front.sepia.ceph.com' reimaging
2023-12-16T00:49:27.501 INFO:teuthology.lock.ops:Updating [smithi005.front.sepia.ceph.com]: reset os type and version on server
2023-12-16T00:49:27.501 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server
2023-12-16T00:49:27.519 INFO:teuthology.lock.ops:Start node 'smithi177.front.sepia.ceph.com' reimaging
2023-12-16T00:49:27.520 INFO:teuthology.lock.ops:Updating [smithi177.front.sepia.ceph.com]: reset os type and version on server
2023-12-16T00:49:27.520 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server
2023-12-16T00:49:27.637 DEBUG:teuthology.provision.fog.smithi005:Requesting image smithi_rhel_8.6 (ID 5)
2023-12-16T00:49:27.660 DEBUG:teuthology.provision.fog.smithi177:Requesting image smithi_rhel_8.6 (ID 5)
2023-12-16T00:49:27.685 INFO:teuthology.provision.fog.smithi005:Scheduling deploy of rhel 8.6
2023-12-16T00:49:27.705 INFO:teuthology.provision.fog.smithi177:Scheduling deploy of rhel 8.6
2023-12-16T00:49:27.932 INFO:teuthology.orchestra.console.smithi005:Power off
2023-12-16T00:49:27.932 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-16T00:49:27.945 INFO:teuthology.orchestra.console.smithi177:Power off
2023-12-16T00:49:27.945 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-16T00:49:27.956 DEBUG:teuthology.orchestra.console.smithi005:power off output: Chassis Power Control: Down/Off
2023-12-16T00:49:27.956 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T00:49:27.967 DEBUG:teuthology.orchestra.console.smithi177:power off output: Chassis Power Control: Down/Off
2023-12-16T00:49:27.967 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T00:49:27.977 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on
2023-12-16T00:49:27.988 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on
2023-12-16T00:49:31.981 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T00:49:31.989 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T00:49:32.091 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on
2023-12-16T00:49:32.097 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on
2023-12-16T00:49:36.096 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T00:49:36.103 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T00:49:36.205 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off
2023-12-16T00:49:36.212 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off
2023-12-16T00:49:36.307 INFO:teuthology.orchestra.console.smithi005:Power off completed
2023-12-16T00:49:36.313 INFO:teuthology.orchestra.console.smithi177:Power off completed
2023-12-16T00:49:36.408 INFO:teuthology.orchestra.console.smithi005:Power on
2023-12-16T00:49:36.409 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-16T00:49:36.416 INFO:teuthology.orchestra.console.smithi177:Power on
2023-12-16T00:49:36.416 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-16T00:49:36.434 DEBUG:teuthology.orchestra.console.smithi005:power on output: Chassis Power Control: Up/On
2023-12-16T00:49:36.434 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T00:49:36.443 DEBUG:teuthology.orchestra.console.smithi177:power on output: Chassis Power Control: Up/On
2023-12-16T00:49:36.443 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T00:49:36.457 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off
2023-12-16T00:49:36.463 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off
2023-12-16T00:49:40.461 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T00:49:40.469 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T00:49:40.571 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on
2023-12-16T00:49:40.577 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off
2023-12-16T00:49:40.672 INFO:teuthology.orchestra.console.smithi005:Power on completed
2023-12-16T00:49:40.773 INFO:teuthology.provision.fog.smithi005:Waiting for deploy to finish
2023-12-16T00:49:44.581 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T00:49:44.690 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on
2023-12-16T00:49:44.790 INFO:teuthology.orchestra.console.smithi177:Power on completed
2023-12-16T00:49:44.891 INFO:teuthology.provision.fog.smithi177:Waiting for deploy to finish
2023-12-16T00:52:15.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-16T00:52:26.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-16T00:52:29.749 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-12-16T00:52:35.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-16T00:52:38.802 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-12-16T00:52:44.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-16T00:52:47.857 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-12-16T00:52:53.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-16T00:52:56.917 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-12-16T00:53:02.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-16T00:53:05.973 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-12-16T00:53:11.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-16T00:53:45.944 ERROR:teuthology.orchestra.connection:Error authenticating with smithi177.front.sepia.ceph.com: AuthenticationException('Authentication timeout.')
2023-12-16T00:53:57.186 ERROR:teuthology.orchestra.connection:Error authenticating with smithi177.front.sepia.ceph.com: BadHostKeyException('smithi177.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fbf95e99250>, <paramiko.ed25519key.Ed25519Key object at 0x7fbf96351700>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi177.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrvFlgKR4VkA5zvnvMXkukiwL30cotoeRtz7QL+SWccOnkwFRqMKsSiBBYuM38a9YQYz8eWp+Pm2zplrPXA6+Y=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIMUZGEspoZd8c3sL+cWh4Jd3KPj8rzmr5T1K/qNR+V8a'
2023-12-16T00:54:04.205 ERROR:teuthology.orchestra.connection:Error authenticating with smithi177.front.sepia.ceph.com: BadHostKeyException('smithi177.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fbf962743a0>, <paramiko.ed25519key.Ed25519Key object at 0x7fbf96351700>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi177.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrvFlgKR4VkA5zvnvMXkukiwL30cotoeRtz7QL+SWccOnkwFRqMKsSiBBYuM38a9YQYz8eWp+Pm2zplrPXA6+Y=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIMUZGEspoZd8c3sL+cWh4Jd3KPj8rzmr5T1K/qNR+V8a'
2023-12-16T00:54:11.977 WARNING:teuthology.provision.fog.smithi005:timed out
2023-12-16T00:54:14.226 ERROR:teuthology.orchestra.connection:Error authenticating with smithi177.front.sepia.ceph.com: BadHostKeyException('smithi177.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fbf96274b80>, <paramiko.ed25519key.Ed25519Key object at 0x7fbf96351700>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi177.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrvFlgKR4VkA5zvnvMXkukiwL30cotoeRtz7QL+SWccOnkwFRqMKsSiBBYuM38a9YQYz8eWp+Pm2zplrPXA6+Y=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIMUZGEspoZd8c3sL+cWh4Jd3KPj8rzmr5T1K/qNR+V8a'
2023-12-16T00:54:15.061 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2023-12-16T00:54:17.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-16T00:54:18.238 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-16T00:54:18.238 DEBUG:teuthology.orchestra.run.smithi005:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-16T00:54:18.503 INFO:teuthology.provision.fog.smithi005:Node is ready
2023-12-16T00:54:18.504 DEBUG:teuthology.orchestra.run.smithi005:> hostname
2023-12-16T00:54:18.524 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005.front.sepia.ceph.com
2023-12-16T00:54:18.525 DEBUG:teuthology.orchestra.run.smithi005:> grep smithi005.front.sepia.ceph.com /etc/hosts
2023-12-16T00:54:18.586 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5 smithi005.front.sepia.ceph.com smithi005
2023-12-16T00:54:18.586 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname smithi005
2023-12-16T00:54:18.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i -e 's/smithi005.front.sepia.ceph.com/smithi005/g' /etc/hosts
2023-12-16T00:54:18.742 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i -e 's/172.21.15.5/172.21.15.5/g' /etc/hosts
2023-12-16T00:54:18.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i -e 's/smithi005.front.sepia.ceph.com/smithi005/g' /etc/hostname
2023-12-16T00:54:18.892 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname smithi005
2023-12-16T00:54:18.966 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release
2023-12-16T00:54:19.027 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="Red Hat Enterprise Linux"
2023-12-16T00:54:19.027 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="8.6 (Ootpa)"
2023-12-16T00:54:19.027 INFO:teuthology.orchestra.run.smithi005.stdout:ID="rhel"
2023-12-16T00:54:19.027 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="fedora"
2023-12-16T00:54:19.028 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="8.6"
2023-12-16T00:54:19.028 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el8"
2023-12-16T00:54:19.028 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-12-16T00:54:19.028 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31"
2023-12-16T00:54:19.028 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-12-16T00:54:19.028 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://www.redhat.com/"
2023-12-16T00:54:19.028 INFO:teuthology.orchestra.run.smithi005.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-12-16T00:54:19.028 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-16T00:54:19.028 INFO:teuthology.orchestra.run.smithi005.stdout:
2023-12-16T00:54:19.028 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-16T00:54:19.028 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-12-16T00:54:19.028 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-12-16T00:54:19.028 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-12-16T00:54:19.029 INFO:teuthology.provision.fog.smithi005:Deploy complete!
2023-12-16T00:54:19.031 DEBUG:teuthology.parallel:result is None
2023-12-16T00:54:27.243 ERROR:teuthology.orchestra.connection:Error authenticating with smithi177.front.sepia.ceph.com: BadHostKeyException('smithi177.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fbf9620b0a0>, <paramiko.ed25519key.Ed25519Key object at 0x7fbf96351700>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi177.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrvFlgKR4VkA5zvnvMXkukiwL30cotoeRtz7QL+SWccOnkwFRqMKsSiBBYuM38a9YQYz8eWp+Pm2zplrPXA6+Y=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIMUZGEspoZd8c3sL+cWh4Jd3KPj8rzmr5T1K/qNR+V8a'
2023-12-16T00:54:43.261 ERROR:teuthology.orchestra.connection:Error authenticating with smithi177.front.sepia.ceph.com: BadHostKeyException('smithi177.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fbf9620baf0>, <paramiko.ed25519key.Ed25519Key object at 0x7fbf96351700>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi177.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrvFlgKR4VkA5zvnvMXkukiwL30cotoeRtz7QL+SWccOnkwFRqMKsSiBBYuM38a9YQYz8eWp+Pm2zplrPXA6+Y=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIMUZGEspoZd8c3sL+cWh4Jd3KPj8rzmr5T1K/qNR+V8a'
2023-12-16T00:55:02.280 ERROR:teuthology.orchestra.connection:Error authenticating with smithi177.front.sepia.ceph.com: BadHostKeyException('smithi177.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fbf963076a0>, <paramiko.ed25519key.Ed25519Key object at 0x7fbf96351700>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi177.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrvFlgKR4VkA5zvnvMXkukiwL30cotoeRtz7QL+SWccOnkwFRqMKsSiBBYuM38a9YQYz8eWp+Pm2zplrPXA6+Y=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIMUZGEspoZd8c3sL+cWh4Jd3KPj8rzmr5T1K/qNR+V8a'
2023-12-16T00:55:24.298 ERROR:teuthology.orchestra.connection:Error authenticating with smithi177.front.sepia.ceph.com: BadHostKeyException('smithi177.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fbf961fb910>, <paramiko.ed25519key.Ed25519Key object at 0x7fbf96351700>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi177.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrvFlgKR4VkA5zvnvMXkukiwL30cotoeRtz7QL+SWccOnkwFRqMKsSiBBYuM38a9YQYz8eWp+Pm2zplrPXA6+Y=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIMUZGEspoZd8c3sL+cWh4Jd3KPj8rzmr5T1K/qNR+V8a'
2023-12-16T00:55:49.316 ERROR:teuthology.orchestra.connection:Error authenticating with smithi177.front.sepia.ceph.com: BadHostKeyException('smithi177.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fbf96212340>, <paramiko.ed25519key.Ed25519Key object at 0x7fbf96351700>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi177.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrvFlgKR4VkA5zvnvMXkukiwL30cotoeRtz7QL+SWccOnkwFRqMKsSiBBYuM38a9YQYz8eWp+Pm2zplrPXA6+Y=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIMUZGEspoZd8c3sL+cWh4Jd3KPj8rzmr5T1K/qNR+V8a'
2023-12-16T00:56:17.334 ERROR:teuthology.orchestra.connection:Error authenticating with smithi177.front.sepia.ceph.com: BadHostKeyException('smithi177.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fbf96212d30>, <paramiko.ed25519key.Ed25519Key object at 0x7fbf96351700>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi177.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrvFlgKR4VkA5zvnvMXkukiwL30cotoeRtz7QL+SWccOnkwFRqMKsSiBBYuM38a9YQYz8eWp+Pm2zplrPXA6+Y=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIMUZGEspoZd8c3sL+cWh4Jd3KPj8rzmr5T1K/qNR+V8a'
2023-12-16T00:56:17.334 WARNING:teuthology.provision.fog.smithi177:'connect to smithi177.front.sepia.ceph.com' reached maximum tries (11) after waiting for 145 seconds
2023-12-16T00:56:23.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-16T00:56:23.576 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-16T00:56:23.576 DEBUG:teuthology.orchestra.run.smithi177:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-16T00:56:23.844 INFO:teuthology.provision.fog.smithi177:Node is ready
2023-12-16T00:56:23.844 DEBUG:teuthology.orchestra.run.smithi177:> hostname
2023-12-16T00:56:23.862 INFO:teuthology.orchestra.run.smithi177.stdout:smithi177
2023-12-16T00:56:23.862 DEBUG:teuthology.orchestra.run.smithi177:> grep smithi177 /etc/hosts
2023-12-16T00:56:23.919 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.15.177 smithi177.front.sepia.ceph.com smithi177
2023-12-16T00:56:23.920 DEBUG:teuthology.orchestra.run.smithi177:> sudo hostname smithi177
2023-12-16T00:56:24.067 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i -e 's/smithi177/smithi177/g' /etc/hosts
2023-12-16T00:56:24.095 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i -e 's/172.21.15.177/172.21.15.177/g' /etc/hosts
2023-12-16T00:56:24.163 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i -e 's/smithi177/smithi177/g' /etc/hostname
2023-12-16T00:56:24.231 DEBUG:teuthology.orchestra.run.smithi177:> sudo hostname smithi177
2023-12-16T00:56:24.296 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release
2023-12-16T00:56:24.353 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="Red Hat Enterprise Linux"
2023-12-16T00:56:24.353 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="8.6 (Ootpa)"
2023-12-16T00:56:24.354 INFO:teuthology.orchestra.run.smithi177.stdout:ID="rhel"
2023-12-16T00:56:24.354 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="fedora"
2023-12-16T00:56:24.354 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="8.6"
2023-12-16T00:56:24.354 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el8"
2023-12-16T00:56:24.354 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-12-16T00:56:24.354 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31"
2023-12-16T00:56:24.354 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-12-16T00:56:24.354 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://www.redhat.com/"
2023-12-16T00:56:24.354 INFO:teuthology.orchestra.run.smithi177.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-12-16T00:56:24.354 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-16T00:56:24.354 INFO:teuthology.orchestra.run.smithi177.stdout:
2023-12-16T00:56:24.354 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-16T00:56:24.354 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-12-16T00:56:24.355 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-12-16T00:56:24.355 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-12-16T00:56:24.355 INFO:teuthology.provision.fog.smithi177:Deploy complete!
2023-12-16T00:56:24.356 DEBUG:teuthology.parallel:result is None
2023-12-16T00:56:24.539 INFO:teuthology.lock.ops:Checking smithi005.front.sepia.ceph.com
2023-12-16T00:56:24.539 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-16T00:56:24.559 INFO:teuthology.lock.ops:Checking smithi177.front.sepia.ceph.com
2023-12-16T00:56:24.560 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-16T00:56:24.585 INFO:teuthology.lock.ops:Updating [smithi005.front.sepia.ceph.com]: set os type and version on server
2023-12-16T00:56:24.585 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com'
2023-12-16T00:56:24.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-16T00:56:24.658 DEBUG:teuthology.orchestra.run.smithi005:> true
2023-12-16T00:56:24.802 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com'
2023-12-16T00:56:24.802 DEBUG:teuthology.orchestra.run.smithi005:> uname -m
2023-12-16T00:56:24.863 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64
2023-12-16T00:56:24.863 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release
2023-12-16T00:56:24.923 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="Red Hat Enterprise Linux"
2023-12-16T00:56:24.924 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="8.6 (Ootpa)"
2023-12-16T00:56:24.924 INFO:teuthology.orchestra.run.smithi005.stdout:ID="rhel"
2023-12-16T00:56:24.924 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="fedora"
2023-12-16T00:56:24.924 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="8.6"
2023-12-16T00:56:24.924 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el8"
2023-12-16T00:56:24.924 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-12-16T00:56:24.924 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31"
2023-12-16T00:56:24.924 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-12-16T00:56:24.924 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://www.redhat.com/"
2023-12-16T00:56:24.924 INFO:teuthology.orchestra.run.smithi005.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-12-16T00:56:24.925 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-16T00:56:24.925 INFO:teuthology.orchestra.run.smithi005.stdout:
2023-12-16T00:56:24.925 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-16T00:56:24.925 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-12-16T00:56:24.925 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-12-16T00:56:24.925 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-12-16T00:56:24.925 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server
2023-12-16T00:56:24.951 INFO:teuthology.lock.ops:Updating [smithi177.front.sepia.ceph.com]: set os type and version on server
2023-12-16T00:56:24.951 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com'
2023-12-16T00:56:24.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-16T00:56:25.019 DEBUG:teuthology.orchestra.run.smithi177:> true
2023-12-16T00:56:25.086 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com'
2023-12-16T00:56:25.087 DEBUG:teuthology.orchestra.run.smithi177:> uname -m
2023-12-16T00:56:25.143 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64
2023-12-16T00:56:25.144 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release
2023-12-16T00:56:25.200 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="Red Hat Enterprise Linux"
2023-12-16T00:56:25.200 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="8.6 (Ootpa)"
2023-12-16T00:56:25.201 INFO:teuthology.orchestra.run.smithi177.stdout:ID="rhel"
2023-12-16T00:56:25.201 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="fedora"
2023-12-16T00:56:25.201 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="8.6"
2023-12-16T00:56:25.201 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el8"
2023-12-16T00:56:25.201 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-12-16T00:56:25.201 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31"
2023-12-16T00:56:25.201 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-12-16T00:56:25.201 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://www.redhat.com/"
2023-12-16T00:56:25.202 INFO:teuthology.orchestra.run.smithi177.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-12-16T00:56:25.202 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-16T00:56:25.202 INFO:teuthology.orchestra.run.smithi177.stdout:
2023-12-16T00:56:25.202 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-16T00:56:25.202 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-12-16T00:56:25.202 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-12-16T00:56:25.202 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-12-16T00:56:25.202 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server
2023-12-16T00:56:25.343 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T00:56:25.475 INFO:teuthology.dispatcher.supervisor:Running job 7493418
2023-12-16T00:56:25.476 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-12-16_00:10:26-upgrade:quincy-x-main-distro-default-smithi/7493418 --name yuriw-2023-12-16_00:10:26-upgrade:quincy-x-main-distro-default-smithi --description upgrade:quincy-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} -- /home/teuthworker/archive/yuriw-2023-12-16_00:10:26-upgrade:quincy-x-main-distro-default-smithi/7493418/orig.config.yaml
2023-12-16T00:56:25.479 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-12-16_00:10:26-upgrade:quincy-x-main-distro-default-smithi/7493418
2023-12-16T00:56:25.479 INFO:teuthology.dispatcher.supervisor:Job PID: 2649443
2023-12-16T00:56:25.480 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-12-16T00:58:25.489 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:00:25.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:02:25.822 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:04:25.986 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:06:26.151 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:08:26.299 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:10:26.466 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:12:26.622 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:14:26.790 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:16:26.953 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:18:27.131 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:20:27.286 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:22:27.450 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:24:27.614 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:26:27.793 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:28:27.958 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:30:28.130 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:32:28.298 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:34:28.462 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:36:28.701 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:38:28.870 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:40:29.034 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:42:29.202 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:44:29.366 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:46:29.534 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:48:29.698 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:50:29.863 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:52:30.030 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:54:30.194 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:56:30.342 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T01:58:30.510 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:00:30.674 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:02:30.841 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:04:31.006 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:06:31.163 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:08:31.326 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:10:31.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:12:31.666 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:14:31.831 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:16:32.043 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:18:32.214 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:20:32.380 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:22:32.546 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:24:32.713 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:26:32.887 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:28:33.051 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:30:33.206 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:32:33.357 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:34:33.522 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:36:33.697 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:38:33.862 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:40:34.026 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:42:34.192 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:44:34.370 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:46:34.536 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:48:34.694 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:50:34.836 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:52:35.006 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:54:35.178 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:56:35.342 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T02:58:35.510 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:00:35.674 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:02:35.842 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:04:36.006 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:06:36.178 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:08:36.346 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:10:36.525 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:12:37.222 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:14:37.398 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:16:37.554 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:18:37.730 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:20:37.898 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:22:38.062 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:24:38.242 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:26:38.411 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:28:38.582 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:30:38.755 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:32:38.922 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:34:39.078 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:36:39.235 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:38:39.402 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:40:39.574 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:42:39.758 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:44:39.930 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:46:40.114 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:48:40.282 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:50:40.442 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:52:40.614 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:54:40.778 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:56:40.958 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T03:58:41.130 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:00:41.299 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:02:41.470 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:04:41.632 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:06:41.800 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:08:41.976 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:10:42.154 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:12:42.314 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:14:42.486 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:16:42.654 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:18:42.822 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:20:42.986 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:22:43.154 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:24:43.319 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:26:43.502 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:28:43.666 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:30:43.830 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:32:43.995 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:34:44.159 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:36:44.326 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:38:44.482 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:40:44.654 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:42:44.822 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:44:45.011 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:46:45.176 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:48:45.341 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:50:45.511 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:52:45.670 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:54:45.838 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:56:46.002 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T04:58:46.162 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:00:46.326 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:02:46.490 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:04:46.678 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:06:46.850 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:08:47.016 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:10:47.178 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:12:47.344 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:14:47.518 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:16:47.686 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:18:47.851 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:20:48.006 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:22:48.174 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:24:48.346 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:26:48.522 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:28:48.702 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:30:48.866 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:32:49.034 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:34:49.206 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:36:49.374 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:38:49.546 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:40:49.718 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:42:49.894 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:44:50.051 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:46:50.218 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:48:50.382 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:50:50.578 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:52:50.709 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:54:50.878 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:56:51.062 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T05:58:51.235 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:00:51.394 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:02:51.558 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:04:51.726 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:06:51.892 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:08:52.054 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:10:52.222 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:12:52.390 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:14:52.572 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:16:52.738 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:18:52.906 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:20:53.078 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:22:53.248 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:24:53.438 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:26:53.602 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:28:53.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:30:53.935 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:32:54.082 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:34:54.250 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:36:54.430 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:38:54.594 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:40:54.758 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:42:54.929 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:44:55.110 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:46:55.278 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:48:55.459 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:50:55.631 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:52:55.814 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:54:55.987 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:56:56.158 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T06:58:56.272 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:00:56.438 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:02:56.607 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:04:57.051 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:06:57.218 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:08:57.386 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:10:57.557 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:12:57.731 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:14:57.896 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:16:58.062 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:18:58.229 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:20:58.410 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:22:58.586 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:24:58.758 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:26:58.926 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:28:59.094 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:30:59.261 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:32:59.442 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:34:59.615 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:36:59.784 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:38:59.946 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:41:00.114 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:43:00.271 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:45:00.434 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:47:00.604 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:49:00.786 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:51:00.964 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:53:01.134 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:55:01.302 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:57:01.478 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T07:59:01.642 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:01:01.810 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:03:01.978 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:05:02.142 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:07:02.302 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:09:02.474 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:11:02.650 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:13:02.814 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:15:02.982 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:17:03.154 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:19:03.318 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:21:03.486 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:23:03.662 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:25:03.834 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:27:04.010 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:29:04.182 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:31:04.366 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:33:04.550 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:35:04.758 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:37:04.934 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:39:05.106 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:41:05.278 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:43:05.454 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:45:05.638 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:47:05.826 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:49:06.006 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:51:06.163 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:53:06.350 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:55:06.538 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:57:06.722 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T08:59:06.902 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:01:07.092 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:03:07.271 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:05:07.424 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:07:07.598 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:09:07.774 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:11:08.010 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:13:08.203 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:15:08.398 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:17:08.578 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:19:08.762 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:21:08.935 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:23:09.110 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:25:09.281 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:27:09.466 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:29:09.646 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:31:09.826 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:33:10.006 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:35:10.186 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:37:10.362 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:39:10.543 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:41:10.734 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:43:10.926 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:45:11.106 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:47:11.273 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:49:11.454 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:51:11.626 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:53:11.811 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:55:11.998 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:57:12.176 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T09:59:12.356 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:01:12.530 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:03:12.702 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:05:12.874 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:07:13.050 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:09:13.175 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:11:13.346 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:13:13.529 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:15:13.710 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:17:13.882 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:19:14.052 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:21:14.317 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:23:14.498 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:25:14.667 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:27:14.848 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:29:15.013 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:31:15.194 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:33:15.370 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:35:15.548 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:37:15.726 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:39:15.898 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:41:16.071 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:43:16.371 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:45:16.550 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:47:16.718 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:49:16.894 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:51:17.074 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:53:17.238 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:55:17.418 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:57:17.606 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T10:59:17.806 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:01:17.970 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:03:18.158 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:05:18.336 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:07:18.511 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:09:18.694 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:11:18.870 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:13:19.046 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:15:19.214 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:17:19.390 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:19:19.566 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:21:19.722 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:23:19.890 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:25:20.062 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:27:20.230 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:29:20.394 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:31:20.555 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:33:20.719 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:35:21.142 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:37:21.310 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:39:21.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:41:21.640 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:43:21.798 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:45:21.962 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:47:22.130 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:49:22.294 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:51:22.462 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:53:22.621 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:55:22.790 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:57:22.958 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T11:59:23.130 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:01:23.298 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:03:23.466 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:05:23.639 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:07:23.814 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:09:23.986 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:11:24.154 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:13:24.320 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:15:24.490 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:17:24.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:19:24.919 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:21:25.090 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:23:25.237 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:25:25.404 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:27:25.578 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:29:25.746 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:31:25.910 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:33:26.068 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:35:26.242 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:37:26.406 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:39:26.566 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:41:26.742 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:43:26.910 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:45:27.062 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:47:27.221 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:49:27.392 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:51:27.557 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:53:27.723 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:55:27.894 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T12:57:28.062 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-12-16T12:57:28.223 INFO:teuthology.kill:Killing Pids: {2649443}
2023-12-16T12:57:29.103 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2023-12-16T12:57:29.103 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b']
2023-12-16T12:57:29.103 INFO:teuthology.misc:Compressing logs...
2023-12-16T12:57:29.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com'
2023-12-16T12:57:29.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-16T12:57:29.224 DEBUG:teuthology.orchestra.run.smithi005:> true
2023-12-16T12:57:29.325 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com'
2023-12-16T12:57:29.325 DEBUG:teuthology.orchestra.run.smithi005:> sudo find /var/lib/ceph/crash -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2023-12-16T12:57:29.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com'
2023-12-16T12:57:29.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-16T12:57:29.482 DEBUG:teuthology.orchestra.run.smithi177:> true
2023-12-16T12:57:29.575 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com'
2023-12-16T12:57:29.576 DEBUG:teuthology.orchestra.run.smithi177:> sudo find /var/lib/ceph/crash -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2023-12-16T12:57:29.697 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-12-16_00:10:26-upgrade:quincy-x-main-distro-default-smithi/7493418/remote/smithi005/crash
2023-12-16T12:57:29.698 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/crash -- .
2023-12-16T12:57:29.754 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-12-16_00:10:26-upgrade:quincy-x-main-distro-default-smithi/7493418/remote/smithi177/crash
2023-12-16T12:57:29.755 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/crash -- .
2023-12-16T12:57:29.825 INFO:teuthology.misc:Compressing logs...
2023-12-16T12:57:29.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo find /home/ubuntu/cephtest/archive -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2023-12-16T12:57:29.828 DEBUG:teuthology.orchestra.run.smithi177:> sudo find /home/ubuntu/cephtest/archive -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2023-12-16T12:57:29.868 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2023-12-16T12:57:29.877 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2023-12-16T12:57:29.879 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2023-12-16T12:57:29.883 INFO:teuthology.orchestra.run.smithi177.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:	/home/ubuntu/cephtest/archive/syslog/kern.log:	 77.9% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2023-12-16T12:57:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2023-12-16T12:57:29.884 INFO:teuthology.orchestra.run.smithi005.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:	/home/ubuntu/cephtest/archive/syslog/kern.log:	 78.1% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2023-12-16T12:57:30.060 INFO:teuthology.orchestra.run.smithi177.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-12-16T12:57:30.060 INFO:teuthology.orchestra.run.smithi177.stderr: 90.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2023-12-16T12:57:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-12-16T12:57:30.214 INFO:teuthology.orchestra.run.smithi005.stderr: 90.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2023-12-16T12:57:30.216 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-12-16_00:10:26-upgrade:quincy-x-main-distro-default-smithi/7493418/remote/smithi005/
2023-12-16T12:57:30.216 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- .
2023-12-16T12:57:30.303 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-12-16_00:10:26-upgrade:quincy-x-main-distro-default-smithi/7493418/remote/smithi177/
2023-12-16T12:57:30.303 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- .
2023-12-16T12:57:30.357 INFO:teuthology.misc:Compressing logs...
2023-12-16T12:57:30.357 DEBUG:teuthology.orchestra.run.smithi005:> sudo find /var/log/ceph -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2023-12-16T12:57:30.360 DEBUG:teuthology.orchestra.run.smithi177:> sudo find /var/log/ceph -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2023-12-16T12:57:30.393 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log
2023-12-16T12:57:30.393 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mon.a.log
2023-12-16T12:57:30.393 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mds.foo.smithi005.kahbhi.log
2023-12-16T12:57:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.1.log
2023-12-16T12:57:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.3.log
2023-12-16T12:57:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.2.log
2023-12-16T12:57:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-volume.log
2023-12-16T12:57:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.0.log
2023-12-16T12:57:30.396 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mon.c.log
2023-12-16T12:57:30.396 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mgr.y.log
2023-12-16T12:57:30.398 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/cephadm.log:	/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mon.a.log:	/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mds.foo.smithi005.kahbhi.log:	/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.1.log:	/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.3.log:	/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.2.log:	/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-volume.log:	/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.0.log:	/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mon.c.log:	 79.8% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mds.foo.smithi005.kahbhi.log.gz
2023-12-16T12:57:30.412 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log
2023-12-16T12:57:30.413 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.5.log
2023-12-16T12:57:30.413 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mds.foo.smithi177.qrgrku.log
2023-12-16T12:57:30.414 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.7.log
2023-12-16T12:57:30.414 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-volume.log
2023-12-16T12:57:30.414 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.6.log
2023-12-16T12:57:30.414 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mon.b.log
2023-12-16T12:57:30.415 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mgr.x.log
2023-12-16T12:57:30.415 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.4.log
2023-12-16T12:57:30.422 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.5.log:	/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mds.foo.smithi177.qrgrku.log:	/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.7.log:	/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-volume.log:	/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.6.log:	/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mon.b.log:	 61.3% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mds.foo.smithi177.qrgrku.log.gz
2023-12-16T12:57:30.446 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mgr.x.log:	/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.4.log:	 91.5% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-volume.log.gz
2023-12-16T12:57:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mgr.y.log:	 91.4% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-volume.log.gz
2023-12-16T12:57:31.005 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/cephadm.log:	 93.2% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mgr.x.log.gz
2023-12-16T12:57:31.354 INFO:teuthology.orchestra.run.smithi005.stderr: 95.9% -- replaced with /var/log/ceph/cephadm.log.gz
2023-12-16T12:57:32.947 INFO:teuthology.orchestra.run.smithi177.stderr: 94.6% -- replaced with /var/log/ceph/cephadm.log.gz
2023-12-16T12:57:45.077 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mgr.y.log: file size changed while zipping
2023-12-16T12:57:45.077 INFO:teuthology.orchestra.run.smithi005.stderr: 89.9% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mgr.y.log.gz
2023-12-16T12:57:47.694 INFO:teuthology.orchestra.run.smithi177.stderr:gzip: /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mon.b.log: file size changed while zipping
2023-12-16T12:57:47.694 INFO:teuthology.orchestra.run.smithi177.stderr: 92.7% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mon.b.log.gz
2023-12-16T12:57:51.200 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mon.c.log: file size changed while zipping
2023-12-16T12:57:51.200 INFO:teuthology.orchestra.run.smithi005.stderr: 92.6% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mon.c.log.gz
2023-12-16T12:58:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mon.a.log: file size changed while zipping
2023-12-16T12:58:33.751 INFO:teuthology.orchestra.run.smithi005.stderr: 90.2% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-mon.a.log.gz
2023-12-16T13:02:21.861 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.2.log: file size changed while zipping
2023-12-16T13:02:21.862 INFO:teuthology.orchestra.run.smithi005.stderr: 93.8% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.2.log.gz
2023-12-16T13:04:53.000 INFO:teuthology.orchestra.run.smithi177.stderr:gzip: /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.6.log: file size changed while zipping
2023-12-16T13:04:53.000 INFO:teuthology.orchestra.run.smithi177.stderr: 93.1% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.6.log.gz
2023-12-16T13:05:16.853 INFO:teuthology.orchestra.run.smithi177.stderr:gzip: /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.5.log: file size changed while zipping
2023-12-16T13:05:16.853 INFO:teuthology.orchestra.run.smithi177.stderr: 93.4% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.5.log.gz
2023-12-16T13:05:36.795 INFO:teuthology.orchestra.run.smithi177.stderr:gzip: /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.4.log: file size changed while zipping
2023-12-16T13:05:36.795 INFO:teuthology.orchestra.run.smithi177.stderr: 92.9% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.4.log.gz
2023-12-16T13:06:02.931 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.3.log: file size changed while zipping
2023-12-16T13:06:02.931 INFO:teuthology.orchestra.run.smithi005.stderr: 93.4% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.3.log.gz
2023-12-16T13:06:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.1.log: file size changed while zipping
2023-12-16T13:06:28.047 INFO:teuthology.orchestra.run.smithi005.stderr: 93.5% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.1.log.gz
2023-12-16T13:06:37.497 INFO:teuthology.orchestra.run.smithi177.stderr:gzip: /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.7.log: file size changed while zipping
2023-12-16T13:06:37.497 INFO:teuthology.orchestra.run.smithi177.stderr: 92.4% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.7.log.gz
2023-12-16T13:06:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.0.log: file size changed while zipping
2023-12-16T13:06:40.150 INFO:teuthology.orchestra.run.smithi005.stderr: 93.0% -- replaced with /var/log/ceph/c1b5157e-9baf-11ee-95a5-87774f69a715/ceph-osd.0.log.gz
2023-12-16T13:06:40.168 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-12-16_00:10:26-upgrade:quincy-x-main-distro-default-smithi/7493418/remote/smithi005/log
2023-12-16T13:06:40.169 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- .
2023-12-16T13:08:19.155 INFO:teuthology.orchestra.run.smithi005.stderr:tar: ./c1b5157e-9baf-11ee-95a5-87774f69a715: file changed as we read it
2023-12-16T13:08:19.168 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-12-16_00:10:26-upgrade:quincy-x-main-distro-default-smithi/7493418/remote/smithi177/log
2023-12-16T13:08:19.169 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- .
2023-12-16T13:10:03.498 INFO:teuthology.kill:No teuthology processes running
2023-12-16T13:10:03.498 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/dispatcher/supervisor.py", line 331, in run_with_watchdog
    kill.kill_job(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/kill.py", line 89, in kill_job
    if kill_processes(run_name, [job_info.get('pid')]):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/kill.py", line 210, in kill_processes
    if survivors:
UnboundLocalError: local variable 'survivors' referenced before assignment
2023-12-16T13:10:03.499 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T13:12:03.670 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-16T13:12:03.746 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-12-16T13:12:03.865 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-12-16T13:12:03.865 INFO:teuthology.nuke:Checking targets against current locks
2023-12-16T13:12:03.913 DEBUG:teuthology.nuke:shortname: smithi005
2023-12-16T13:12:03.914 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-16T13:12:03.927 DEBUG:teuthology.nuke:shortname: smithi177
2023-12-16T13:12:03.927 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-16T13:12:03.934 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-16_00:10:26-upgrade:quincy-x-main-distro-default-smithi/7493418', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-16 00:49:26.497836', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJXymoss7uZyoaNSQX4VornbM5ISLhBNLycQ/Viq8nejnrLg6NJSChI1mTFh6Blhw00T6WGBR+1eCDdwx9cG9Cg='}
2023-12-16T13:12:03.947 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-16_00:10:26-upgrade:quincy-x-main-distro-default-smithi/7493418', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-16 00:49:26.496072', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrvFlgKR4VkA5zvnvMXkukiwL30cotoeRtz7QL+SWccOnkwFRqMKsSiBBYuM38a9YQYz8eWp+Pm2zplrPXA6+Y='}
2023-12-16T13:12:03.976 INFO:teuthology.orchestra.console.smithi005:Power off
2023-12-16T13:12:03.976 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-16T13:12:03.998 INFO:teuthology.orchestra.console.smithi177:Power off
2023-12-16T13:12:03.998 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-16T13:12:04.008 DEBUG:teuthology.orchestra.console.smithi005:power off output: Chassis Power Control: Down/Off
2023-12-16T13:12:04.008 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T13:12:04.022 DEBUG:teuthology.orchestra.console.smithi177:power off output: Chassis Power Control: Down/Off
2023-12-16T13:12:04.023 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T13:12:04.032 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on
2023-12-16T13:12:04.048 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on
2023-12-16T13:12:08.035 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T13:12:08.050 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T13:12:08.145 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on
2023-12-16T13:12:08.158 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on
2023-12-16T13:12:12.149 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T13:12:12.159 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-16T13:12:12.259 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off
2023-12-16T13:12:12.268 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off
2023-12-16T13:12:12.361 INFO:teuthology.orchestra.console.smithi005:Power off completed
2023-12-16T13:12:12.369 INFO:teuthology.orchestra.console.smithi177:Power off completed
2023-12-16T13:12:12.508 INFO:teuthology.lock.ops:unlocked: smithi005.front.sepia.ceph.com
2023-12-16T13:12:12.516 INFO:teuthology.lock.ops:unlocked: smithi177.front.sepia.ceph.com