2024-04-13T19:10:02.281 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-13T19:10:02.281 DEBUG:teuthology.run:Teuthology command: teuthology --description powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-bitmap powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_kernel_untar_build thrashosds-health} --name teuthology-2024-04-11_22:32:02-powercycle-reef-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-04-11_22:32:02-powercycle-reef-distro-default-smithi/7653257 --verbose --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-04-11_22:32:02-powercycle-reef-distro-default-smithi/7653257/orig.config.yaml 2024-04-13T19:10:02.316 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T19:10:02.406 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-11_22:32:02-powercycle-reef-distro-default-smithi/7653257 branch: reef description: powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-bitmap powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_kernel_untar_build thrashosds-health} email: ceph-qa@ceph.com first_in_suite: false job_id: '7653257' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-04-11_22:32:02-powercycle-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: reef ceph: conf: client: fuse_default_permissions: 0 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(MDS_TRIM\) - \(MDS_SLOW_REQUEST\) - MDS_SLOW_METADATA_IO - Behind on trimming - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: c96c8d478b644b6433700141c5f69a1a68f8121b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c96c8d478b644b6433700141c5f69a1a68f8121b thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: 4f70a9a90881107996c8b7a3e89ba8b32624713c owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.b - mon.c - mgr.x - mgr.y - mds.a - client.0 - - osd.0 - - osd.1 - - osd.2 seed: 3031 sha1: c96c8d478b644b6433700141c5f69a1a68f8121b sleep_before_teardown: 0 subset: 1/4 suite: powercycle suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_4f70a9a90881107996c8b7a3e89ba8b32624713c/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 4f70a9a90881107996c8b7a3e89ba8b32624713c targets: smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO66b6kggSbH/Vun2Ik/CXDLylu8JmFuHpFdvJ5OIhjf25rWGT+uMtTf+91Tm66Kf5OcrAzgRgU8O+//WWZFc1Y= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCfLUogE/wMnQIvMbWGrk91Ae91TZ6yf2JP/flYc/+DQsaBsQe/oiJcxEApP0egaxj9yvrDZtBj8yfP0ND71sWw= smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMqmjkHqUi4Cyq0d5XIWL6rxnkSjKCiwtJn8OCdXE4A9fdbmmc+8saj5YOxAsgtGZ4CbyMg60ZG6Cjj70Ew7CJo= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCRIQ1vqCdsOPuYSZKH3OzmhPoZA4MTEbrbw1RRNeZc6gNbsWOUIilMsWRMZl7puYRr3QvtGMRbhHhNcKepkR3o= tasks: - install: extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - thrashosds: chance_down: 1.0 powercycle: true timeout: 600 - ceph-fuse: null - workunit: clients: all: - kernel_untar_build.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-11_22:32:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-13T19:10:02.406 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_4f70a9a90881107996c8b7a3e89ba8b32624713c/qa; will attempt to use it 2024-04-13T19:10:02.407 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_4f70a9a90881107996c8b7a3e89ba8b32624713c/qa/tasks 2024-04-13T19:10:02.407 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-13T19:10:02.414 INFO:teuthology.task.internal:Checking packages... 2024-04-13T19:10:02.438 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'c96c8d478b644b6433700141c5f69a1a68f8121b' 2024-04-13T19:10:02.438 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-13T19:10:02.439 INFO:teuthology.packaging:ref: None 2024-04-13T19:10:02.439 INFO:teuthology.packaging:tag: None 2024-04-13T19:10:02.439 INFO:teuthology.packaging:branch: reef 2024-04-13T19:10:02.439 INFO:teuthology.packaging:sha1: c96c8d478b644b6433700141c5f69a1a68f8121b 2024-04-13T19:10:02.439 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef 2024-04-13T19:10:02.759 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1219-ge6cbcc46-1jammy 2024-04-13T19:10:02.761 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-13T19:10:02.767 INFO:teuthology.task.internal:no buildpackages task found 2024-04-13T19:10:02.768 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-13T19:10:02.780 INFO:teuthology.task.internal:Saving configuration 2024-04-13T19:10:02.794 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-13T19:10:02.800 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-13T19:10:02.826 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-11_22:32:02-powercycle-reef-distro-default-smithi/7653257', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 19:04:26.029818', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPTWsPBcT7iiidTv95lOESlKgsRIvrzqVfX/rEJOMUZg'} 2024-04-13T19:10:02.843 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-11_22:32:02-powercycle-reef-distro-default-smithi/7653257', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 19:04:26.031347', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINHmR5/ul9rKkPuBX3EaPurYffW0alQcwrk/9rapQ1sl'} 2024-04-13T19:10:02.862 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-11_22:32:02-powercycle-reef-distro-default-smithi/7653257', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 19:04:26.028065', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJPd+mRbyaEp3wd3km4mNRXbTTcovEHuihSQsJ52dEll'} 2024-04-13T19:10:02.879 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-11_22:32:02-powercycle-reef-distro-default-smithi/7653257', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 19:04:26.026156', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIH8nNNMZGEb0CXvmtiOrfaz4tldgZcY7a4NphMm+s9n4'} 2024-04-13T19:10:02.879 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-13T19:10:02.885 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'client.0'] 2024-04-13T19:10:02.885 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['osd.0'] 2024-04-13T19:10:02.885 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['osd.1'] 2024-04-13T19:10:02.885 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['osd.2'] 2024-04-13T19:10:02.885 INFO:teuthology.run_tasks:Running task console_log... 2024-04-13T19:10:03.007 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f030c2a4280>, signals=[15]) 2024-04-13T19:10:03.008 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-13T19:10:03.013 INFO:teuthology.task.internal:Opening connections... 2024-04-13T19:10:03.014 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2024-04-13T19:10:03.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T19:10:03.100 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-04-13T19:10:03.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T19:10:03.102 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 180, in connect rem.connect() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 387, in connect self.ssh = connection.connect(**args) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 409, in connect raise NoValidConnectionsError(errors) paramiko.ssh_exception.NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-13T19:10:03.319 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=ae77fb4be94742409f9534370d0911fa Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 180, in connect rem.connect() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 387, in connect self.ssh = connection.connect(**args) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 409, in connect raise NoValidConnectionsError(errors) paramiko.ssh_exception.NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-13T19:10:03.321 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-13T19:10:03.334 INFO:teuthology.nuke:Checking targets against current locks 2024-04-13T19:10:03.372 DEBUG:teuthology.nuke:shortname: smithi038 2024-04-13T19:10:03.372 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-13T19:10:03.397 DEBUG:teuthology.nuke:shortname: smithi142 2024-04-13T19:10:03.397 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-13T19:10:03.402 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-11_22:32:02-powercycle-reef-distro-default-smithi/7653257', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 19:04:26.029818', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPTWsPBcT7iiidTv95lOESlKgsRIvrzqVfX/rEJOMUZg'} 2024-04-13T19:10:03.427 DEBUG:teuthology.nuke:shortname: smithi146 2024-04-13T19:10:03.428 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-13T19:10:03.434 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-11_22:32:02-powercycle-reef-distro-default-smithi/7653257', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 19:04:26.031347', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINHmR5/ul9rKkPuBX3EaPurYffW0alQcwrk/9rapQ1sl'} 2024-04-13T19:10:03.450 DEBUG:teuthology.nuke:shortname: smithi164 2024-04-13T19:10:03.451 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-13T19:10:03.457 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-11_22:32:02-powercycle-reef-distro-default-smithi/7653257', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 19:04:26.028065', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJPd+mRbyaEp3wd3km4mNRXbTTcovEHuihSQsJ52dEll'} 2024-04-13T19:10:03.466 INFO:teuthology.orchestra.console.smithi038:Power off 2024-04-13T19:10:03.467 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T19:10:03.480 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-11_22:32:02-powercycle-reef-distro-default-smithi/7653257', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 19:04:26.026156', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIH8nNNMZGEb0CXvmtiOrfaz4tldgZcY7a4NphMm+s9n4'} 2024-04-13T19:10:03.489 DEBUG:teuthology.orchestra.console.smithi038:power off output: Chassis Power Control: Down/Off 2024-04-13T19:10:03.490 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T19:10:03.512 INFO:teuthology.orchestra.console.smithi142:Power off 2024-04-13T19:10:03.512 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T19:10:03.525 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-04-13T19:10:03.534 DEBUG:teuthology.orchestra.console.smithi142:power off output: Chassis Power Control: Down/Off 2024-04-13T19:10:03.534 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T19:10:03.566 INFO:teuthology.orchestra.console.smithi146:Power off 2024-04-13T19:10:03.566 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T19:10:03.589 INFO:teuthology.orchestra.console.smithi164:Power off 2024-04-13T19:10:03.590 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T19:10:03.599 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-04-13T19:10:03.616 DEBUG:teuthology.orchestra.console.smithi146:power off output: Chassis Power Control: Down/Off 2024-04-13T19:10:03.616 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T19:10:03.632 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off 2024-04-13T19:10:03.632 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T19:10:03.672 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2024-04-13T19:10:03.684 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-04-13T19:10:07.526 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T19:10:07.600 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T19:10:07.636 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-04-13T19:10:07.673 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T19:10:07.685 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T19:10:07.714 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is off 2024-04-13T19:10:07.783 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2024-04-13T19:10:07.795 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2024-04-13T19:10:07.816 INFO:teuthology.orchestra.console.smithi142:Power off completed 2024-04-13T19:10:07.884 INFO:teuthology.orchestra.console.smithi146:Power off completed 2024-04-13T19:10:07.896 INFO:teuthology.orchestra.console.smithi164:Power off completed 2024-04-13T19:10:11.638 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T19:10:11.748 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-04-13T19:10:15.752 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T19:10:15.863 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off 2024-04-13T19:10:15.964 INFO:teuthology.orchestra.console.smithi038:Power off completed 2024-04-13T19:10:16.068 INFO:teuthology.run:Summary data: description: powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-bitmap powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_kernel_untar_build thrashosds-health} failure_reason: '[Errno None] Unable to connect to port 22 on 172.21.15.142' owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=ae77fb4be94742409f9534370d0911fa status: fail success: false 2024-04-13T19:10:16.069 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T19:10:16.154 INFO:teuthology.run:FAIL