2023-12-04T12:46:03.369 INFO:root:teuthology version: 0.0.1.dev214+gf2de108
2023-12-04T12:46:03.370 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T12:46:03.485 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7efcba931460>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7efcbb85b550>, signals=[15])
2023-12-04T12:46:03.485 INFO:teuthology.lock.ops:Start node 'smithi195.front.sepia.ceph.com' reimaging
2023-12-04T12:46:03.486 INFO:teuthology.lock.ops:Updating [smithi195.front.sepia.ceph.com]: reset os type and version on server
2023-12-04T12:46:03.486 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server
2023-12-04T12:46:03.616 DEBUG:teuthology.provision.fog.smithi195:Requesting image smithi_centos_9.stream (ID 10)
2023-12-04T12:46:03.671 INFO:teuthology.provision.fog.smithi195:Scheduling deploy of centos 9.stream
2023-12-04T12:46:04.091 INFO:teuthology.orchestra.console.smithi195:Power off
2023-12-04T12:46:04.091 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-04T12:46:04.121 DEBUG:teuthology.orchestra.console.smithi195:power off output: Chassis Power Control: Down/Off
2023-12-04T12:46:04.122 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T12:46:04.144 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on
2023-12-04T12:46:08.149 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T12:46:08.257 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on
2023-12-04T12:46:12.260 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T12:46:12.368 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on
2023-12-04T12:46:16.370 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T12:46:16.478 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off
2023-12-04T12:46:16.578 INFO:teuthology.orchestra.console.smithi195:Power off completed
2023-12-04T12:46:16.679 INFO:teuthology.orchestra.console.smithi195:Power on
2023-12-04T12:46:16.679 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-04T12:46:16.701 DEBUG:teuthology.orchestra.console.smithi195:power on output: Chassis Power Control: Up/On
2023-12-04T12:46:16.701 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T12:46:16.721 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off
2023-12-04T12:46:20.725 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T12:46:20.834 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off
2023-12-04T12:46:24.837 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T12:46:24.945 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on
2023-12-04T12:46:25.046 INFO:teuthology.orchestra.console.smithi195:Power on completed
2023-12-04T12:46:25.146 INFO:teuthology.provision.fog.smithi195:Waiting for deploy to finish
2023-12-04T12:49:41.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T12:49:45.026 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195
2023-12-04T12:49:51.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T12:49:54.114 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195
2023-12-04T12:50:00.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T12:50:03.202 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195
2023-12-04T12:50:09.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T12:50:12.258 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195
2023-12-04T12:50:18.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T12:50:21.346 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195
2023-12-04T12:50:27.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T12:50:30.402 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195
2023-12-04T12:50:36.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T12:51:36.466 WARNING:teuthology.provision.fog.smithi195:timed out
2023-12-04T12:51:42.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T12:51:42.742 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-04T12:51:42.743 DEBUG:teuthology.orchestra.run.smithi195:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-04T12:51:43.091 INFO:teuthology.provision.fog.smithi195:Node is ready
2023-12-04T12:51:43.092 DEBUG:teuthology.orchestra.run.smithi195:> hostname
2023-12-04T12:51:43.106 INFO:teuthology.orchestra.run.smithi195.stdout:smithi195.front.sepia.ceph.com
2023-12-04T12:51:43.107 DEBUG:teuthology.orchestra.run.smithi195:> grep smithi195.front.sepia.ceph.com /etc/hosts
2023-12-04T12:51:43.163 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.15.195 smithi195.front.sepia.ceph.com smithi195
2023-12-04T12:51:43.164 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname smithi195
2023-12-04T12:51:43.268 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/smithi195.front.sepia.ceph.com/smithi195/g' /etc/hosts
2023-12-04T12:51:43.296 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/172.21.15.195/172.21.15.195/g' /etc/hosts
2023-12-04T12:51:43.364 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/smithi195.front.sepia.ceph.com/smithi195/g' /etc/hostname
2023-12-04T12:51:43.431 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname smithi195
2023-12-04T12:51:43.497 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release
2023-12-04T12:51:43.551 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream"
2023-12-04T12:51:43.552 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9"
2023-12-04T12:51:43.552 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos"
2023-12-04T12:51:43.552 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora"
2023-12-04T12:51:43.552 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9"
2023-12-04T12:51:43.552 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9"
2023-12-04T12:51:43.552 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9"
2023-12-04T12:51:43.552 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31"
2023-12-04T12:51:43.552 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon"
2023-12-04T12:51:43.552 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-12-04T12:51:43.552 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/"
2023-12-04T12:51:43.552 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-04T12:51:43.552 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-12-04T12:51:43.552 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-04T12:51:43.553 INFO:teuthology.provision.fog.smithi195:Deploy complete!
2023-12-04T12:51:43.554 DEBUG:teuthology.parallel:result is None
2023-12-04T12:51:43.669 INFO:teuthology.lock.ops:Checking smithi195.front.sepia.ceph.com
2023-12-04T12:51:43.670 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-04T12:51:43.692 INFO:teuthology.lock.ops:Updating [smithi195.front.sepia.ceph.com]: set os type and version on server
2023-12-04T12:51:43.692 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com'
2023-12-04T12:51:43.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T12:51:43.779 DEBUG:teuthology.orchestra.run.smithi195:> true
2023-12-04T12:51:43.802 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com'
2023-12-04T12:51:43.802 DEBUG:teuthology.orchestra.run.smithi195:> uname -m
2023-12-04T12:51:43.857 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64
2023-12-04T12:51:43.857 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release
2023-12-04T12:51:43.911 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream"
2023-12-04T12:51:43.911 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9"
2023-12-04T12:51:43.911 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos"
2023-12-04T12:51:43.911 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora"
2023-12-04T12:51:43.911 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9"
2023-12-04T12:51:43.911 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9"
2023-12-04T12:51:43.911 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9"
2023-12-04T12:51:43.911 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31"
2023-12-04T12:51:43.911 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon"
2023-12-04T12:51:43.912 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-12-04T12:51:43.912 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/"
2023-12-04T12:51:43.912 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-04T12:51:43.912 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-12-04T12:51:43.912 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-04T12:51:43.912 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server
2023-12-04T12:51:43.932 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T12:51:44.013 INFO:teuthology.dispatcher.supervisor:Running job 7477139
2023-12-04T12:51:44.013 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/bin/teuthology -v --owner scheduled_matan@teuthology --archive /home/teuthworker/archive/matan-2023-12-04_12:43:05-crimson-rados-wip-matanb-crimson-snaptrimevent-lifetime-v2-distro-crimson-smithi/7477139 --name matan-2023-12-04_12:43:05-crimson-rados-wip-matanb-crimson-snaptrimevent-lifetime-v2-distro-crimson-smithi --description crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/sample_fio} -- /home/teuthworker/archive/matan-2023-12-04_12:43:05-crimson-rados-wip-matanb-crimson-snaptrimevent-lifetime-v2-distro-crimson-smithi/7477139/orig.config.yaml
2023-12-04T12:51:44.016 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/matan-2023-12-04_12:43:05-crimson-rados-wip-matanb-crimson-snaptrimevent-lifetime-v2-distro-crimson-smithi/7477139
2023-12-04T12:51:44.016 INFO:teuthology.dispatcher.supervisor:Job PID: 504297
2023-12-04T12:51:44.017 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-12-04T12:53:44.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T12:55:44.295 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T12:57:44.459 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T12:59:44.630 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:01:44.801 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:03:44.971 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:05:45.136 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:07:45.299 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:09:45.454 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:11:45.616 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:13:45.783 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:15:45.937 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:17:46.064 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:19:46.229 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:21:46.391 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:23:46.544 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:25:46.706 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:27:46.828 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:29:46.992 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:31:47.159 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:33:47.274 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:35:47.431 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:37:47.587 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:39:47.723 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:41:47.893 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:43:48.063 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:45:48.223 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:47:48.386 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:49:48.542 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:51:48.645 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:53:48.794 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:55:48.961 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:57:49.093 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T13:59:49.260 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:01:49.431 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:03:49.568 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:05:49.719 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:07:49.891 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:09:50.055 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:11:50.208 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:13:50.377 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:15:50.550 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:17:50.712 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:19:50.878 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:21:51.034 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:23:51.204 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:25:51.366 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:27:51.523 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:29:51.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:31:51.852 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:33:52.010 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:35:52.150 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:37:52.289 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:39:52.451 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:41:52.609 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:43:52.775 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:45:52.937 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:47:53.091 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:49:53.261 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:51:53.506 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:53:53.673 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:55:53.835 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:57:53.993 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T14:59:54.139 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:01:54.292 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:03:54.427 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:05:54.591 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:07:54.729 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:09:54.884 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:11:55.055 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:13:55.137 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:15:55.286 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:17:55.427 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:19:55.575 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:21:55.738 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:23:55.896 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:25:56.059 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:27:56.223 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:29:56.392 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:31:56.552 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:33:56.720 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:35:56.794 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:37:56.956 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:39:57.140 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:41:57.308 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:43:57.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:45:57.655 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:47:57.834 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:49:58.019 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:51:58.195 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:53:58.363 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:55:58.532 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:57:58.696 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T15:59:58.860 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:01:59.018 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:03:59.183 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:05:59.347 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:07:59.516 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:09:59.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:11:59.856 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:14:00.031 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:16:00.203 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:18:00.382 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:20:00.543 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:22:00.707 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:24:00.891 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:26:01.026 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:28:01.187 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:30:01.388 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:32:01.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:34:01.698 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:36:01.863 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:38:02.032 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:40:02.316 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:42:02.474 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:44:02.644 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:46:02.826 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:48:02.968 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:50:03.134 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:52:03.327 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:54:03.452 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:56:03.623 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T16:58:03.783 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:00:03.926 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:02:04.060 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:04:04.208 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:06:04.350 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:08:04.498 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:10:04.647 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:12:04.787 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:14:04.950 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:16:05.088 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:18:05.253 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:20:05.401 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:22:05.506 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:24:05.682 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:26:05.854 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:28:06.061 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:30:06.242 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:32:06.416 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:34:06.570 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:36:06.667 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:38:06.737 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:40:06.888 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:42:07.047 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:44:07.198 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:46:07.347 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:48:07.496 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:50:07.621 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:52:07.741 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:54:07.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:56:08.063 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T17:58:08.232 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:00:08.364 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:02:08.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:04:08.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:06:08.839 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:08:08.981 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:10:09.136 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:12:09.313 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:14:09.473 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:16:09.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:18:09.605 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:20:09.775 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:22:09.942 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:24:10.115 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:26:10.267 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:28:10.436 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:30:10.601 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:32:10.748 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:34:10.912 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:36:11.013 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:38:11.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:40:11.331 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:42:11.434 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:44:11.590 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:46:11.708 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:48:11.867 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:50:12.035 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:52:12.212 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:54:12.384 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:56:12.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T18:58:12.739 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:00:12.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:02:13.096 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:04:13.269 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:06:13.433 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:08:13.600 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:10:13.759 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:12:13.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:14:14.163 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:16:14.327 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:18:14.497 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:20:14.660 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:22:14.817 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:24:14.918 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:26:20.087 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:28:20.218 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:30:29.243 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:32:29.411 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:34:29.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:36:34.723 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:38:39.064 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:40:44.200 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:42:44.337 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:44:49.509 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:46:53.768 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:49:03.951 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:51:09.123 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:53:12.752 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:55:14.159 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:57:14.325 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T19:59:19.511 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:01:24.016 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:03:29.199 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:05:29.363 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:07:34.020 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:09:39.126 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:11:44.291 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:13:49.458 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:15:54.608 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:17:55.176 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:20:00.344 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:22:03.267 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:24:08.440 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:26:08.562 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:28:13.735 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:30:18.917 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:32:19.137 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:34:24.292 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:36:29.472 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:38:34.637 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:40:34.801 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:42:39.911 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:44:45.015 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:46:45.187 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:48:50.364 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:50:50.531 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:52:50.611 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:54:55.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:57:00.979 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T20:59:03.303 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:01:08.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:03:13.631 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:05:18.676 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:07:23.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:09:29.068 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:11:33.539 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:13:38.704 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:15:43.876 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:17:44.048 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:19:44.199 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:21:44.365 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:23:44.523 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:25:44.687 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:27:49.859 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:29:55.027 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:32:00.179 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:34:05.311 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:36:10.487 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:38:10.586 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:40:10.677 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:42:15.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:44:21.025 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:46:21.173 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:48:21.332 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:50:21.447 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:52:26.592 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:54:31.747 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:56:36.860 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T21:58:41.948 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:00:42.076 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:02:42.227 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:04:47.397 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:06:52.552 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:08:52.731 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:10:52.953 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:12:56.505 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:15:01.748 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:17:06.936 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:19:07.099 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:21:12.296 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:23:12.420 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:25:13.542 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:27:13.697 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:29:13.871 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:31:14.040 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:33:19.198 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:35:19.375 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:37:19.552 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:39:24.718 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:41:24.850 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:43:30.011 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:45:30.211 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:45:35.162 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/matan-2023-12-04_12:43:05-crimson-rados-wip-matanb-crimson-snaptrimevent-lifetime-v2-distro-crimson-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2023-12-04T22:47:36.456 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:49:41.720 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:51:46.951 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:53:47.120 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:55:52.207 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:57:52.371 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T22:59:52.485 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:01:52.647 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:03:52.812 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:05:52.953 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:07:56.755 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:10:01.845 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:12:01.996 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:14:07.129 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:16:12.256 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:18:12.418 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:20:17.581 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:22:22.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:24:27.866 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:26:33.002 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:28:38.131 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:30:43.248 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:32:48.409 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:34:48.579 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:36:48.746 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:38:48.911 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:40:54.079 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:42:59.228 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:44:59.388 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:46:59.539 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:49:04.613 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:51:09.733 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:53:14.933 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:55:15.156 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:57:18.975 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T23:59:24.155 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:01:24.323 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:03:24.504 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:05:24.767 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:07:24.919 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:09:25.063 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:11:25.227 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:13:25.410 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:15:25.644 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:17:25.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:19:25.957 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:21:26.144 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:23:26.350 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:25:26.459 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:27:26.643 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:29:26.810 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:31:26.897 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:33:27.004 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:35:27.174 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:37:27.381 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:39:27.562 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:41:27.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:43:27.947 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:45:28.144 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:47:28.505 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:49:28.691 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:51:28.831 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:53:29.000 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-12-05T00:53:29.118 INFO:teuthology.kill:Killing Pids: {504297}
2023-12-05T00:53:29.957 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']
2023-12-05T00:53:29.958 INFO:teuthology.misc:Compressing logs...
2023-12-05T00:53:29.958 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com'
2023-12-05T00:53:29.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-05T00:53:30.107 DEBUG:teuthology.orchestra.run.smithi195:> true
2023-12-05T00:53:30.177 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com'
2023-12-05T00:53:30.177 DEBUG:teuthology.orchestra.run.smithi195:> sudo find /var/lib/ceph/crash -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2023-12-05T00:53:30.251 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2023-12-04_12:43:05-crimson-rados-wip-matanb-crimson-snaptrimevent-lifetime-v2-distro-crimson-smithi/7477139/remote/smithi195/crash
2023-12-05T00:53:30.252 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/crash -- .
2023-12-05T00:53:30.315 INFO:teuthology.misc:Compressing logs...
2023-12-05T00:53:30.316 DEBUG:teuthology.orchestra.run.smithi195:> sudo find /home/ubuntu/cephtest/archive -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2023-12-05T00:53:30.383 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2023-12-05T00:53:30.383 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2023-12-05T00:53:30.383 INFO:teuthology.orchestra.run.smithi195.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:	  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2023-12-05T00:53:30.383 INFO:teuthology.orchestra.run.smithi195.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:	  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2023-12-05T00:53:30.384 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2023-12-04_12:43:05-crimson-rados-wip-matanb-crimson-snaptrimevent-lifetime-v2-distro-crimson-smithi/7477139/remote/smithi195/
2023-12-05T00:53:30.384 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- .
2023-12-05T00:53:30.464 INFO:teuthology.misc:Compressing logs...
2023-12-05T00:53:30.464 DEBUG:teuthology.orchestra.run.smithi195:> sudo find /var/log/ceph -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2023-12-05T00:53:30.541 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82424.log
2023-12-05T00:53:30.542 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83344.log
2023-12-05T00:53:30.542 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log
2023-12-05T00:53:30.542 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.82424.log:	 50.0% -- replaced with /var/log/ceph/ceph-client.admin.82424.log.gz
2023-12-05T00:53:30.542 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83344.log:	 50.0% -- replaced with /var/log/ceph/ceph-client.admin.83344.log.gz
2023-12-05T00:53:30.542 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.80787.log
2023-12-05T00:53:30.542 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83447.log
2023-12-05T00:53:30.542 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mon.a.log:	/var/log/ceph/ceph.tmp-client.admin.80787.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86820.log
2023-12-05T00:53:30.542 INFO:teuthology.orchestra.run.smithi195.stderr: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.80787.log.gz
2023-12-05T00:53:30.542 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.83447.log:	 -- /var/log/ceph/ceph-client.admin.82234.log
2023-12-05T00:53:30.543 INFO:teuthology.orchestra.run.smithi195.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.83447.log.gz
2023-12-05T00:53:30.543 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.86820.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82384.log
2023-12-05T00:53:30.543 INFO:teuthology.orchestra.run.smithi195.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.86820.log.gz
2023-12-05T00:53:30.543 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.82234.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86961.log
2023-12-05T00:53:30.543 INFO:teuthology.orchestra.run.smithi195.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.82234.log.gz
2023-12-05T00:53:30.543 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.82384.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83117.log
2023-12-05T00:53:30.543 INFO:teuthology.orchestra.run.smithi195.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz
2023-12-05T00:53:30.543 INFO:teuthology.orchestra.run.smithi195.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.82384.log.gz
2023-12-05T00:53:30.543 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.86961.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86936.log
2023-12-05T00:53:30.543 INFO:teuthology.orchestra.run.smithi195.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.86961.log.gz
2023-12-05T00:53:30.543 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83117.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86882.log
2023-12-05T00:53:30.543 INFO:teuthology.orchestra.run.smithi195.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.83117.log.gz
2023-12-05T00:53:30.544 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.86936.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87031.log
2023-12-05T00:53:30.544 INFO:teuthology.orchestra.run.smithi195.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.86936.log.gz
2023-12-05T00:53:30.544 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.86882.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83342.log
2023-12-05T00:53:30.544 INFO:teuthology.orchestra.run.smithi195.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.86882.log.gz
2023-12-05T00:53:30.544 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.87031.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82582.log
2023-12-05T00:53:30.544 INFO:teuthology.orchestra.run.smithi195.stderr: 65.6% -- replaced with /var/log/ceph/ceph-client.admin.87031.log.gz
2023-12-05T00:53:30.544 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83342.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82612.log
2023-12-05T00:53:30.544 INFO:teuthology.orchestra.run.smithi195.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.83342.log.gz
2023-12-05T00:53:30.544 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.82582.log:	gzip -5 --verbose -- /var/log/ceph/ceph.audit.log
2023-12-05T00:53:30.544 INFO:teuthology.orchestra.run.smithi195.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.82582.log.gz
2023-12-05T00:53:30.545 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.82612.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83450.log
2023-12-05T00:53:30.545 INFO:teuthology.orchestra.run.smithi195.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.82612.log.gz
2023-12-05T00:53:30.545 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph.audit.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83262.log
2023-12-05T00:53:30.545 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83450.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83556.log
2023-12-05T00:53:30.545 INFO:teuthology.orchestra.run.smithi195.stderr: 86.2% -- replaced with /var/log/ceph/ceph.audit.log.gz
2023-12-05T00:53:30.545 INFO:teuthology.orchestra.run.smithi195.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.83450.log.gz
2023-12-05T00:53:30.545 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83262.log:	gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log
2023-12-05T00:53:30.545 INFO:teuthology.orchestra.run.smithi195.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.83262.log.gz
2023-12-05T00:53:30.545 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83556.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83598.log
2023-12-05T00:53:30.545 INFO:teuthology.orchestra.run.smithi195.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.83556.log.gz
2023-12-05T00:53:30.545 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83019.log
2023-12-05T00:53:30.546 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.1.log:	/var/log/ceph/ceph-client.admin.83598.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82146.log
2023-12-05T00:53:30.546 INFO:teuthology.orchestra.run.smithi195.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.83598.log.gz
2023-12-05T00:53:30.546 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83019.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87007.log
2023-12-05T00:53:30.546 INFO:teuthology.orchestra.run.smithi195.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.83019.log.gz
2023-12-05T00:53:30.546 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.82146.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86984.log
2023-12-05T00:53:30.546 INFO:teuthology.orchestra.run.smithi195.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.82146.log.gz
2023-12-05T00:53:30.546 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.87007.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86911.log
2023-12-05T00:53:30.546 INFO:teuthology.orchestra.run.smithi195.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.87007.log.gz
2023-12-05T00:53:30.546 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.86984.log:	gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log
2023-12-05T00:53:30.546 INFO:teuthology.orchestra.run.smithi195.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.86984.log.gz
2023-12-05T00:53:30.546 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.86911.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83178.log
2023-12-05T00:53:30.547 INFO:teuthology.orchestra.run.smithi195.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.86911.log.gz
2023-12-05T00:53:30.547 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mgr.x.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83343.log
2023-12-05T00:53:30.547 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83178.log:	 50.0% -- replaced with /var/log/ceph/ceph-client.admin.83178.log.gz
2023-12-05T00:53:30.547 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83075.log
2023-12-05T00:53:30.547 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83470.log
2023-12-05T00:53:30.547 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83343.log:	 50.0% -- replaced with /var/log/ceph/ceph-client.admin.83343.log.gz
2023-12-05T00:53:30.547 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83075.log:	 50.0% -- replaced with /var/log/ceph/ceph-client.admin.83075.log.gz
2023-12-05T00:53:30.547 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log
2023-12-05T00:53:30.547 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log
2023-12-05T00:53:30.547 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83470.log:	 49.6% -- replaced with /var/log/ceph/ceph-client.admin.83470.log.gz
2023-12-05T00:53:30.547 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.0.log:	gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log
2023-12-05T00:53:30.547 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82344.log
2023-12-05T00:53:30.548 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.2.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83640.log
2023-12-05T00:53:30.548 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.82344.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83220.log
2023-12-05T00:53:30.548 INFO:teuthology.orchestra.run.smithi195.stderr: 89.4% -- replaced with /var/log/ceph/ceph.log.gz
2023-12-05T00:53:30.548 INFO:teuthology.orchestra.run.smithi195.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.82344.log.gz
2023-12-05T00:53:30.548 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83640.log:	 50.0% -- replaced with /var/log/ceph/ceph-client.admin.83640.log.gz
2023-12-05T00:53:30.548 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83220.log:	 49.2% -- replaced with /var/log/ceph/ceph-client.admin.83220.log.gz
2023-12-05T00:53:30.560 INFO:teuthology.orchestra.run.smithi195.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz
2023-12-05T00:53:30.620 INFO:teuthology.orchestra.run.smithi195.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz
2023-12-05T00:53:30.627 INFO:teuthology.orchestra.run.smithi195.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz
2023-12-05T00:53:30.634 INFO:teuthology.orchestra.run.smithi195.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz
2023-12-05T00:53:30.636 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/matan-2023-12-04_12:43:05-crimson-rados-wip-matanb-crimson-snaptrimevent-lifetime-v2-distro-crimson-smithi/7477139/remote/smithi195/log
2023-12-05T00:53:30.637 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- .
2023-12-05T00:53:30.860 INFO:teuthology.kill:No teuthology processes running
2023-12-05T00:53:30.860 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/dispatcher/supervisor.py", line 331, in run_with_watchdog
    kill.kill_job(
  File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/kill.py", line 89, in kill_job
    if kill_processes(run_name, [job_info.get('pid')]):
  File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/kill.py", line 210, in kill_processes
    if survivors:
UnboundLocalError: local variable 'survivors' referenced before assignment
2023-12-05T00:53:30.861 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:55:30.976 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-05T00:55:31.062 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-12-05T00:55:31.159 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-12-05T00:55:31.159 INFO:teuthology.nuke:Checking targets against current locks
2023-12-05T00:55:31.247 DEBUG:teuthology.nuke:shortname: smithi195
2023-12-05T00:55:31.247 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-05T00:55:31.279 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-12-04_12:43:05-crimson-rados-wip-matanb-crimson-snaptrimevent-lifetime-v2-distro-crimson-smithi/7477139', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-04 12:46:02.534399', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDc9Gg5S7kqdC9DIQFenIRiTVK7Dl3Bez4aGoCA7jg11LpQYTY6OOrepNFMz/lHDLHsAjGZ3OwDchbRWNTRs0p4='}
2023-12-05T00:55:31.373 INFO:teuthology.orchestra.console.smithi195:Power off
2023-12-05T00:55:31.373 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-05T00:55:31.422 DEBUG:teuthology.orchestra.console.smithi195:power off output: Chassis Power Control: Down/Off
2023-12-05T00:55:31.423 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-05T00:55:31.457 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on
2023-12-05T00:55:35.462 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-05T00:55:35.574 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on
2023-12-05T00:55:39.578 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-05T00:55:39.687 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off
2023-12-05T00:55:39.788 INFO:teuthology.orchestra.console.smithi195:Power off completed
2023-12-05T00:55:39.960 INFO:teuthology.lock.ops:unlocked: smithi195.front.sepia.ceph.com