2024-09-15T20:54:56.105 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T20:54:56.119 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T20:54:56.197 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905658 branch: main description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/crush} email: ceph-qa@ceph.com first_in_suite: false job_id: '7905658' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: main ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 workunit: branch: main sha1: 7498493279982dfe87f00616198a5967475f6169 owner: scheduled_teuthology@teuthology priority: 101 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 2855 sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 sleep_before_teardown: 0 subset: 17936/100000 suite: rados suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7498493279982dfe87f00616198a5967475f6169 targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCSg5UV89ON8oU/xKCRXPpKU9rEUGMK2Gh7CXXaEJQDAT69LA7hxR8bKXEitGLqRHj7+ZcU/maoX6Yz9q/Zw+EA= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - crush teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_20:00:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T20:54:56.197 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa; will attempt to use it 2024-09-15T20:54:56.198 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa/tasks 2024-09-15T20:54:56.198 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T20:54:56.200 INFO:teuthology.task.internal:Checking packages... 2024-09-15T20:54:56.221 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'fc3e0bff7468fd55478adbae7f75508b749124b6' 2024-09-15T20:54:56.221 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T20:54:56.221 INFO:teuthology.packaging:ref: None 2024-09-15T20:54:56.221 INFO:teuthology.packaging:tag: None 2024-09-15T20:54:56.221 INFO:teuthology.packaging:branch: main 2024-09-15T20:54:56.221 INFO:teuthology.packaging:sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T20:54:56.222 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=main 2024-09-15T20:54:56.429 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T20:54:56.430 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T20:54:56.437 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T20:54:56.438 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T20:54:56.444 INFO:teuthology.task.internal:Saving configuration 2024-09-15T20:54:56.452 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T20:54:56.461 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T20:54:56.486 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905658', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 20:49:34.300420', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJHZgwuhHbg1J/Y/FxMXEDBQk7xWISkrmzaETHPyJidP'} 2024-09-15T20:54:56.486 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T20:54:56.492 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2024-09-15T20:54:56.492 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T20:54:56.525 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe1a8503520>, signals=[15]) 2024-09-15T20:54:56.525 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T20:54:56.531 INFO:teuthology.task.internal:Opening connections... 2024-09-15T20:54:56.531 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-09-15T20:54:56.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:54:56.612 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T20:54:56.618 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-09-15T20:54:56.733 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-09-15T20:54:56.734 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-09-15T20:54:56.780 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-15T20:54:56.780 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Ubuntu" 2024-09-15T20:54:56.780 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="22.04" 2024-09-15T20:54:56.780 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-15T20:54:56.780 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_CODENAME=jammy 2024-09-15T20:54:56.780 INFO:teuthology.orchestra.run.smithi016.stdout:ID=ubuntu 2024-09-15T20:54:56.780 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE=debian 2024-09-15T20:54:56.780 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-15T20:54:56.781 INFO:teuthology.orchestra.run.smithi016.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-15T20:54:56.781 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-15T20:54:56.781 INFO:teuthology.orchestra.run.smithi016.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-15T20:54:56.781 INFO:teuthology.orchestra.run.smithi016.stdout:UBUNTU_CODENAME=jammy 2024-09-15T20:54:56.781 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-09-15T20:54:56.803 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T20:54:56.811 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T20:54:56.816 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T20:54:56.816 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-09-15T20:54:56.828 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T20:54:56.834 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T20:54:56.834 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-09-15T20:54:56.878 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T20:54:56.910 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T20:54:56.934 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T20:54:56.934 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T20:54:56.934 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T20:54:56.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:54:56.940 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-15T20:54:56.988 INFO:teuthology.orchestra.run.smithi016.stdout:5.15.0-56-generic 2024-09-15T20:54:56.989 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.15.0-56-generic 2024-09-15T20:54:56.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo apt-get clean 2024-09-15T20:54:57.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo apt-get update 2024-09-15T20:54:57.254 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T20:54:57.264 INFO:teuthology.orchestra.run.smithi016.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-15T20:54:57.264 INFO:teuthology.orchestra.run.smithi016.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-15T20:54:57.331 INFO:teuthology.orchestra.run.smithi016.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-15T20:54:57.497 INFO:teuthology.orchestra.run.smithi016.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-15T20:54:57.576 INFO:teuthology.orchestra.run.smithi016.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-15T20:54:57.594 INFO:teuthology.orchestra.run.smithi016.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-15T20:54:57.600 INFO:teuthology.orchestra.run.smithi016.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-15T20:54:57.604 INFO:teuthology.orchestra.run.smithi016.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-15T20:54:57.604 INFO:teuthology.orchestra.run.smithi016.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-15T20:54:57.605 INFO:teuthology.orchestra.run.smithi016.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-15T20:54:57.630 INFO:teuthology.orchestra.run.smithi016.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-15T20:54:57.633 INFO:teuthology.orchestra.run.smithi016.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-15T20:54:57.634 INFO:teuthology.orchestra.run.smithi016.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-15T20:54:57.640 INFO:teuthology.orchestra.run.smithi016.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-15T20:54:57.648 INFO:teuthology.orchestra.run.smithi016.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-15T20:54:57.664 INFO:teuthology.orchestra.run.smithi016.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-15T20:54:57.685 INFO:teuthology.orchestra.run.smithi016.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-15T20:54:57.685 INFO:teuthology.orchestra.run.smithi016.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-15T20:54:57.685 INFO:teuthology.orchestra.run.smithi016.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-15T20:54:57.709 INFO:teuthology.orchestra.run.smithi016.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-15T20:54:57.709 INFO:teuthology.orchestra.run.smithi016.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-15T20:54:57.709 INFO:teuthology.orchestra.run.smithi016.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-15T20:54:57.709 INFO:teuthology.orchestra.run.smithi016.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-15T20:54:57.710 INFO:teuthology.orchestra.run.smithi016.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-15T20:54:57.710 INFO:teuthology.orchestra.run.smithi016.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-15T20:54:57.710 INFO:teuthology.orchestra.run.smithi016.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-15T20:54:57.711 INFO:teuthology.orchestra.run.smithi016.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-15T20:54:57.711 INFO:teuthology.orchestra.run.smithi016.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-15T20:54:57.711 INFO:teuthology.orchestra.run.smithi016.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-15T20:54:57.711 INFO:teuthology.orchestra.run.smithi016.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-15T20:54:57.723 INFO:teuthology.orchestra.run.smithi016.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-15T20:54:57.724 INFO:teuthology.orchestra.run.smithi016.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-15T20:54:57.726 INFO:teuthology.orchestra.run.smithi016.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-15T20:54:57.726 INFO:teuthology.orchestra.run.smithi016.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-15T20:54:57.732 INFO:teuthology.orchestra.run.smithi016.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-15T20:54:57.736 INFO:teuthology.orchestra.run.smithi016.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-15T20:54:57.737 INFO:teuthology.orchestra.run.smithi016.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-15T20:54:57.741 INFO:teuthology.orchestra.run.smithi016.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-15T20:54:57.743 INFO:teuthology.orchestra.run.smithi016.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-15T20:54:57.743 INFO:teuthology.orchestra.run.smithi016.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-15T20:54:57.743 INFO:teuthology.orchestra.run.smithi016.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-15T20:54:57.743 INFO:teuthology.orchestra.run.smithi016.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-15T20:54:57.743 INFO:teuthology.orchestra.run.smithi016.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-15T20:55:04.440 INFO:teuthology.orchestra.run.smithi016.stdout:Fetched 16.0 MB in 2s (7,083 kB/s) 2024-09-15T20:55:05.958 INFO:teuthology.orchestra.run.smithi016.stdout:Reading package lists... 2024-09-15T20:55:05.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T20:55:06.039 INFO:teuthology.orchestra.run.smithi016.stdout:Reading package lists... 2024-09-15T20:55:06.255 INFO:teuthology.orchestra.run.smithi016.stdout:Building dependency tree... 2024-09-15T20:55:06.255 INFO:teuthology.orchestra.run.smithi016.stdout:Reading state information... 2024-09-15T20:55:06.452 INFO:teuthology.orchestra.run.smithi016.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T20:55:06.453 INFO:teuthology.orchestra.run.smithi016.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T20:55:06.453 INFO:teuthology.orchestra.run.smithi016.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T20:55:06.453 INFO:teuthology.orchestra.run.smithi016.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T20:55:06.453 INFO:teuthology.orchestra.run.smithi016.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-15T20:55:06.453 INFO:teuthology.orchestra.run.smithi016.stdout: python2.7-minimal 2024-09-15T20:55:06.453 INFO:teuthology.orchestra.run.smithi016.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T20:55:06.453 INFO:teuthology.orchestra.run.smithi016.stdout:The following additional packages will be installed: 2024-09-15T20:55:06.454 INFO:teuthology.orchestra.run.smithi016.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-15T20:55:06.454 INFO:teuthology.orchestra.run.smithi016.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-15T20:55:06.454 INFO:teuthology.orchestra.run.smithi016.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-15T20:55:06.456 INFO:teuthology.orchestra.run.smithi016.stdout:Suggested packages: 2024-09-15T20:55:06.456 INFO:teuthology.orchestra.run.smithi016.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-15T20:55:06.456 INFO:teuthology.orchestra.run.smithi016.stdout:Recommended packages: 2024-09-15T20:55:06.456 INFO:teuthology.orchestra.run.smithi016.stdout: thermald 2024-09-15T20:55:06.502 INFO:teuthology.orchestra.run.smithi016.stdout:The following NEW packages will be installed: 2024-09-15T20:55:06.502 INFO:teuthology.orchestra.run.smithi016.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-15T20:55:06.502 INFO:teuthology.orchestra.run.smithi016.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-15T20:55:06.503 INFO:teuthology.orchestra.run.smithi016.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-15T20:55:06.504 INFO:teuthology.orchestra.run.smithi016.stdout:The following packages will be upgraded: 2024-09-15T20:55:06.504 INFO:teuthology.orchestra.run.smithi016.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-15T20:55:06.685 INFO:teuthology.orchestra.run.smithi016.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-15T20:55:06.685 INFO:teuthology.orchestra.run.smithi016.stdout:Need to get 113 MB of archives. 2024-09-15T20:55:06.685 INFO:teuthology.orchestra.run.smithi016.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-15T20:55:06.685 INFO:teuthology.orchestra.run.smithi016.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-15T20:55:08.099 INFO:teuthology.orchestra.run.smithi016.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-15T20:55:08.453 INFO:teuthology.orchestra.run.smithi016.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-15T20:55:10.640 INFO:teuthology.orchestra.run.smithi016.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-15T20:55:10.640 INFO:teuthology.orchestra.run.smithi016.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-15T20:55:10.640 INFO:teuthology.orchestra.run.smithi016.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-15T20:55:10.992 INFO:teuthology.orchestra.run.smithi016.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-15T20:55:11.072 INFO:teuthology.orchestra.run.smithi016.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-15T20:55:11.512 INFO:teuthology.orchestra.run.smithi016.stdout:Fetched 113 MB in 5s (24.8 MB/s) 2024-09-15T20:55:11.713 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-15T20:55:13.886 INFO:teuthology.orchestra.run.smithi016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-15T20:55:13.890 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:55:14.077 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:55:16.279 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-15T20:55:16.300 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:55:16.400 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:55:16.932 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-15T20:55:16.952 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:55:16.990 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:55:24.283 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T20:55:24.401 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T20:55:24.759 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T20:55:24.913 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T20:55:25.232 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-15T20:55:25.255 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-15T20:55:25.299 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-15T20:55:31.844 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-15T20:55:31.881 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:55:31.919 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:55:35.750 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T20:55:35.889 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T20:55:36.264 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-15T20:55:36.369 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:55:36.528 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-15T20:55:36.664 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:55:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-15T20:55:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-15T20:55:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-15T20:55:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-15T20:55:38.226 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:55:40.119 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:55:41.545 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-15T20:55:41.679 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-15T20:55:41.822 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:55:41.913 INFO:teuthology.orchestra.run.smithi016.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-15T20:55:41.913 INFO:teuthology.orchestra.run.smithi016.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-15T20:55:57.004 INFO:teuthology.orchestra.run.smithi016.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-15T20:55:57.004 INFO:teuthology.orchestra.run.smithi016.stdout:Sourcing file `/etc/default/grub' 2024-09-15T20:55:57.027 INFO:teuthology.orchestra.run.smithi016.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T20:55:57.045 INFO:teuthology.orchestra.run.smithi016.stdout:Generating grub configuration file ... 2024-09-15T20:55:57.392 INFO:teuthology.orchestra.run.smithi016.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-15T20:55:57.411 INFO:teuthology.orchestra.run.smithi016.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-15T20:55:57.743 INFO:teuthology.orchestra.run.smithi016.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-15T20:55:57.750 INFO:teuthology.orchestra.run.smithi016.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-15T20:55:57.825 INFO:teuthology.orchestra.run.smithi016.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-15T20:55:57.830 INFO:teuthology.orchestra.run.smithi016.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-15T20:55:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-15T20:55:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-15T20:55:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-15T20:55:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:done 2024-09-15T20:55:58.601 DEBUG:teuthology.orchestra.run.smithi016:> dpkg -s linux-image-generic 2024-09-15T20:55:58.624 INFO:teuthology.orchestra.run.smithi016.stdout:Package: linux-image-generic 2024-09-15T20:55:58.624 INFO:teuthology.orchestra.run.smithi016.stdout:Status: install ok installed 2024-09-15T20:55:58.624 INFO:teuthology.orchestra.run.smithi016.stdout:Priority: optional 2024-09-15T20:55:58.625 INFO:teuthology.orchestra.run.smithi016.stdout:Section: kernel 2024-09-15T20:55:58.625 INFO:teuthology.orchestra.run.smithi016.stdout:Installed-Size: 21 2024-09-15T20:55:58.625 INFO:teuthology.orchestra.run.smithi016.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T20:55:58.625 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: amd64 2024-09-15T20:55:58.625 INFO:teuthology.orchestra.run.smithi016.stdout:Source: linux-meta 2024-09-15T20:55:58.625 INFO:teuthology.orchestra.run.smithi016.stdout:Version: 5.15.0.121.121 2024-09-15T20:55:58.626 INFO:teuthology.orchestra.run.smithi016.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-15T20:55:58.626 INFO:teuthology.orchestra.run.smithi016.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T20:55:58.626 INFO:teuthology.orchestra.run.smithi016.stdout:Recommends: thermald 2024-09-15T20:55:58.626 INFO:teuthology.orchestra.run.smithi016.stdout:Description: Generic Linux kernel image 2024-09-15T20:55:58.626 INFO:teuthology.orchestra.run.smithi016.stdout: This package will always depend on the latest generic kernel image 2024-09-15T20:55:58.626 INFO:teuthology.orchestra.run.smithi016.stdout: available. 2024-09-15T20:55:58.627 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-15T20:55:58.627 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T20:55:58.627 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T20:55:58.628 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-15T20:55:58.628 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T20:55:58.628 DEBUG:teuthology.orchestra.run.smithi016:> sudo apt-get clean 2024-09-15T20:55:58.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo apt-get update 2024-09-15T20:55:59.015 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-15T20:55:59.117 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T20:55:59.200 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-15T20:55:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-15T20:56:00.501 INFO:teuthology.orchestra.run.smithi016.stdout:Reading package lists... 2024-09-15T20:56:00.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T20:56:00.587 INFO:teuthology.orchestra.run.smithi016.stdout:Reading package lists... 2024-09-15T20:56:00.800 INFO:teuthology.orchestra.run.smithi016.stdout:Building dependency tree... 2024-09-15T20:56:00.801 INFO:teuthology.orchestra.run.smithi016.stdout:Reading state information... 2024-09-15T20:56:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-15T20:56:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T20:56:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T20:56:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T20:56:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T20:56:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-15T20:56:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-15T20:56:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T20:56:01.035 INFO:teuthology.orchestra.run.smithi016.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-15T20:56:01.037 DEBUG:teuthology.orchestra.run.smithi016:> dpkg -s linux-image-generic 2024-09-15T20:56:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:Package: linux-image-generic 2024-09-15T20:56:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:Status: install ok installed 2024-09-15T20:56:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:Priority: optional 2024-09-15T20:56:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:Section: kernel 2024-09-15T20:56:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:Installed-Size: 21 2024-09-15T20:56:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T20:56:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: amd64 2024-09-15T20:56:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:Source: linux-meta 2024-09-15T20:56:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:Version: 5.15.0.121.121 2024-09-15T20:56:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-15T20:56:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T20:56:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:Recommends: thermald 2024-09-15T20:56:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:Description: Generic Linux kernel image 2024-09-15T20:56:01.055 INFO:teuthology.orchestra.run.smithi016.stdout: This package will always depend on the latest generic kernel image 2024-09-15T20:56:01.055 INFO:teuthology.orchestra.run.smithi016.stdout: available. 2024-09-15T20:56:01.055 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-09-15T20:56:01.099 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.ACknYXxEAw 2024-09-15T20:56:01.100 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ACknYXxEAw 2024-09-15T20:56:01.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.ACknYXxEAw 2024-09-15T20:56:01.267 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.ACknYXxEAw is 10KB 2024-09-15T20:56:01.280 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.ACknYXxEAw 2024-09-15T20:56:01.286 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-15T20:56:01.339 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T20:56:01.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-15T20:56:01.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-15T20:56:01.426 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-15T20:56:01.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo update-grub 2024-09-15T20:56:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:Sourcing file `/etc/default/grub' 2024-09-15T20:56:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T20:56:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-09-15T20:56:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-15T20:56:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-15T20:56:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-15T20:56:03.405 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-15T20:56:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-15T20:56:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-15T20:56:03.569 INFO:teuthology.orchestra.run.smithi016.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-15T20:56:03.569 INFO:teuthology.orchestra.run.smithi016.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-15T20:56:03.570 INFO:teuthology.orchestra.run.smithi016.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-15T20:56:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-09-15T20:56:03.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-09-15T20:56:33.656 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-15T20:56:33.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-15T20:56:33.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:56:52.029 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-09-15T20:57:01.037 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-15T20:57:01.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:58:01.098 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T20:58:13.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-15T20:58:13.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:58:13.326 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-15T20:58:14.020 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-15T20:58:14.021 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-15T20:58:14.021 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-15T20:58:14.069 INFO:teuthology.orchestra.run.smithi016.stdout:5.15.0-121-generic 2024-09-15T20:58:14.069 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-15T20:58:14.069 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T20:58:14.069 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-15T20:58:15.070 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-15T20:58:15.070 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T20:58:15.139 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2024-09-15T20:58:15.159 DEBUG:teuthology.parallel:result is None 2024-09-15T20:58:15.159 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T20:58:15.166 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T20:58:15.167 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T20:58:15.173 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T20:58:15.179 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T20:58:15.191 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T20:58:15.191 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T20:58:15.230 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T20:58:15.237 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T20:58:15.237 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T20:58:15.285 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T20:58:15.293 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T20:58:15.295 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T20:58:15.302 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T20:58:15.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T20:58:15.349 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T20:58:15.358 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T20:58:15.359 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T20:58:15.397 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T20:58:15.445 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T20:58:15.493 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T20:58:15.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T20:58:15.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-09-15T20:58:15.637 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T20:58:15.646 INFO:teuthology.task.internal:Starting timer... 2024-09-15T20:58:15.646 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T20:58:15.659 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T20:58:15.667 DEBUG:teuthology.task.selinux:Excluding smithi016: OS 'ubuntu' does not support SELinux 2024-09-15T20:58:15.668 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T20:58:15.668 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-15T20:58:15.668 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T20:58:15.668 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T20:58:15.677 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-15T20:58:15.693 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-15T20:58:15.936 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T20:58:15.947 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T20:58:15.948 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T21:05:56.510 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com')] 2024-09-15T21:05:56.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-15T21:05:56.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:05:56.598 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-15T21:05:56.677 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-15T21:05:56.677 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T21:05:56.689 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T21:05:56.690 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:05:56.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:05:56.746 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-15T21:05:56.746 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: Command line: ntpd -gq 2024-09-15T21:05:56.746 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: ---------------------------------------------------- 2024-09-15T21:05:56.746 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: ntp-4 is maintained by Network Time Foundation, 2024-09-15T21:05:56.746 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-15T21:05:56.746 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: corporation. Support and training for ntp-4 are 2024-09-15T21:05:56.746 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: available at https://www.nwtime.org/support 2024-09-15T21:05:56.746 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: ---------------------------------------------------- 2024-09-15T21:05:56.747 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: proto: precision = 0.046 usec (-24) 2024-09-15T21:05:56.747 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: basedate set to 2022-02-04 2024-09-15T21:05:56.747 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: gps base set to 2022-02-06 (week 2196) 2024-09-15T21:05:56.747 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T21:05:56.748 INFO:teuthology.orchestra.run.smithi016.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T21:05:56.748 INFO:teuthology.orchestra.run.smithi016.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-15T21:05:56.749 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: restrict ::: KOD does nothing without LIMITED. 2024-09-15T21:05:56.749 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: Listen and drop on 0 v6wildcard [::]:123 2024-09-15T21:05:56.749 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-15T21:05:56.749 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: Listen normally on 2 lo 127.0.0.1:123 2024-09-15T21:05:56.749 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: Listen normally on 3 ens1f0 172.21.15.16:123 2024-09-15T21:05:56.749 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: Listen normally on 4 lo [::1]:123 2024-09-15T21:05:56.749 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1564%4]:123 2024-09-15T21:05:56.749 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:05:56 ntpd[18356]: Listening on routing socket on fd #22 for interface updates 2024-09-15T21:05:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:57 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:05:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:57 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:05:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:57 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:05:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:57 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:05:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:57 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:05:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:57 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:05:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:57 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:05:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:57 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:05:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:05:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:05:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:05:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:05:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:05:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:05:59 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:06:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:06:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:06:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:06:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:06:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:06:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:06:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:06:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:06:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:06:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:06:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:06:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:06:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:06:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:06:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:06:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:06:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:06:01.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:01 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:06:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:06:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:06:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:06:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:06:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:06:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:06:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:06:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:06:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:06:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:06:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:06:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:06:03.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:06:03.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:06:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:15 Sep 21:06:03 ntpd[18356]: ntpd: time slew +0.000015 s 2024-09-15T21:06:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:ntpd: time slew +0.000015s 2024-09-15T21:06:03.749 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-15T21:06:03.750 INFO:teuthology.orchestra.run.smithi016.stderr:15 Sep 21:06:03 ntpd[18356]: can't open /var/log/ntpstats/loopstats.20240915: Permission denied 2024-09-15T21:06:03.803 INFO:teuthology.orchestra.run.smithi016.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T21:06:03.804 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================== 2024-09-15T21:06:03.804 INFO:teuthology.orchestra.run.smithi016.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:06:03.804 INFO:teuthology.orchestra.run.smithi016.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:06:03.804 INFO:teuthology.orchestra.run.smithi016.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:06:03.804 INFO:teuthology.orchestra.run.smithi016.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:06:03.805 INFO:teuthology.run_tasks:Running task install... 2024-09-15T21:06:03.814 DEBUG:teuthology.task.install:project ceph 2024-09-15T21:06:03.814 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}} 2024-09-15T21:06:03.814 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'} 2024-09-15T21:06:03.814 INFO:teuthology.task.install:Using flavor: default 2024-09-15T21:06:03.820 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T21:06:03.820 INFO:teuthology.task.install:extra packages: [] 2024-09-15T21:06:03.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo apt-key list | grep Ceph 2024-09-15T21:06:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-15T21:06:03.931 INFO:teuthology.orchestra.run.smithi016.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-15T21:06:03.932 INFO:teuthology.orchestra.run.smithi016.stdout:uid [ unknown] Ceph.com (release key) 2024-09-15T21:06:03.933 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-15T21:06:03.933 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:06:04.075 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default/ 2024-09-15T21:06:04.075 INFO:teuthology.task.install.deb:Package version is 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:06:04.186 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T21:06:04.186 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-15T21:06:04.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo apt-get update 2024-09-15T21:06:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T21:06:04.382 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-15T21:06:04.387 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-15T21:06:04.399 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-15T21:06:04.428 INFO:teuthology.orchestra.run.smithi016.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy InRelease 2024-09-15T21:06:04.452 INFO:teuthology.orchestra.run.smithi016.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-15T21:06:04.474 INFO:teuthology.orchestra.run.smithi016.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-15T21:06:04.495 INFO:teuthology.orchestra.run.smithi016.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,427 B] 2024-09-15T21:06:04.522 INFO:teuthology.orchestra.run.smithi016.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-15T21:06:04.736 INFO:teuthology.orchestra.run.smithi016.stdout:Fetched 29.2 kB in 0s (83.9 kB/s) 2024-09-15T21:06:05.769 INFO:teuthology.orchestra.run.smithi016.stdout:Reading package lists... 2024-09-15T21:06:05.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4930-gfc3e0bff-1jammy cephadm=19.3.0-4930-gfc3e0bff-1jammy ceph-mds=19.3.0-4930-gfc3e0bff-1jammy ceph-mgr=19.3.0-4930-gfc3e0bff-1jammy ceph-common=19.3.0-4930-gfc3e0bff-1jammy ceph-fuse=19.3.0-4930-gfc3e0bff-1jammy ceph-test=19.3.0-4930-gfc3e0bff-1jammy ceph-volume=19.3.0-4930-gfc3e0bff-1jammy radosgw=19.3.0-4930-gfc3e0bff-1jammy python3-rados=19.3.0-4930-gfc3e0bff-1jammy python3-rgw=19.3.0-4930-gfc3e0bff-1jammy python3-cephfs=19.3.0-4930-gfc3e0bff-1jammy python3-rbd=19.3.0-4930-gfc3e0bff-1jammy libcephfs2=19.3.0-4930-gfc3e0bff-1jammy libcephfs-dev=19.3.0-4930-gfc3e0bff-1jammy librados2=19.3.0-4930-gfc3e0bff-1jammy librbd1=19.3.0-4930-gfc3e0bff-1jammy rbd-fuse=19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:06:05.843 INFO:teuthology.orchestra.run.smithi016.stdout:Reading package lists... 2024-09-15T21:06:06.033 INFO:teuthology.orchestra.run.smithi016.stdout:Building dependency tree... 2024-09-15T21:06:06.033 INFO:teuthology.orchestra.run.smithi016.stdout:Reading state information... 2024-09-15T21:06:06.225 INFO:teuthology.orchestra.run.smithi016.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T21:06:06.225 INFO:teuthology.orchestra.run.smithi016.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T21:06:06.225 INFO:teuthology.orchestra.run.smithi016.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T21:06:06.225 INFO:teuthology.orchestra.run.smithi016.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T21:06:06.226 INFO:teuthology.orchestra.run.smithi016.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-15T21:06:06.226 INFO:teuthology.orchestra.run.smithi016.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-15T21:06:06.226 INFO:teuthology.orchestra.run.smithi016.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T21:06:06.227 INFO:teuthology.orchestra.run.smithi016.stdout:The following additional packages will be installed: 2024-09-15T21:06:06.227 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-15T21:06:06.228 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-15T21:06:06.228 INFO:teuthology.orchestra.run.smithi016.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-15T21:06:06.228 INFO:teuthology.orchestra.run.smithi016.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-15T21:06:06.228 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-15T21:06:06.228 INFO:teuthology.orchestra.run.smithi016.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-15T21:06:06.228 INFO:teuthology.orchestra.run.smithi016.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-15T21:06:06.228 INFO:teuthology.orchestra.run.smithi016.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-15T21:06:06.228 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-15T21:06:06.228 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-15T21:06:06.228 INFO:teuthology.orchestra.run.smithi016.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-15T21:06:06.228 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-15T21:06:06.228 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-15T21:06:06.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-15T21:06:06.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-15T21:06:06.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-15T21:06:06.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-15T21:06:06.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-15T21:06:06.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-15T21:06:06.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-15T21:06:06.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-15T21:06:06.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-15T21:06:06.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-15T21:06:06.229 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet zip 2024-09-15T21:06:06.230 INFO:teuthology.orchestra.run.smithi016.stdout:Suggested packages: 2024-09-15T21:06:06.230 INFO:teuthology.orchestra.run.smithi016.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-15T21:06:06.230 INFO:teuthology.orchestra.run.smithi016.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-15T21:06:06.230 INFO:teuthology.orchestra.run.smithi016.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-15T21:06:06.230 INFO:teuthology.orchestra.run.smithi016.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-15T21:06:06.230 INFO:teuthology.orchestra.run.smithi016.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-15T21:06:06.230 INFO:teuthology.orchestra.run.smithi016.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-15T21:06:06.230 INFO:teuthology.orchestra.run.smithi016.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-15T21:06:06.230 INFO:teuthology.orchestra.run.smithi016.stdout:Recommended packages: 2024-09-15T21:06:06.230 INFO:teuthology.orchestra.run.smithi016.stdout: btrfs-tools 2024-09-15T21:06:06.338 INFO:teuthology.orchestra.run.smithi016.stdout:The following NEW packages will be installed: 2024-09-15T21:06:06.338 INFO:teuthology.orchestra.run.smithi016.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-15T21:06:06.338 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-15T21:06:06.339 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-15T21:06:06.339 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-15T21:06:06.339 INFO:teuthology.orchestra.run.smithi016.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-15T21:06:06.339 INFO:teuthology.orchestra.run.smithi016.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-15T21:06:06.340 INFO:teuthology.orchestra.run.smithi016.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-15T21:06:06.340 INFO:teuthology.orchestra.run.smithi016.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-15T21:06:06.340 INFO:teuthology.orchestra.run.smithi016.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-15T21:06:06.340 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-15T21:06:06.341 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-15T21:06:06.341 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-15T21:06:06.341 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-15T21:06:06.341 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-15T21:06:06.341 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-15T21:06:06.341 INFO:teuthology.orchestra.run.smithi016.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-15T21:06:06.341 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-15T21:06:06.341 INFO:teuthology.orchestra.run.smithi016.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-15T21:06:06.341 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-15T21:06:06.341 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-15T21:06:06.341 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-15T21:06:06.341 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-15T21:06:06.341 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-15T21:06:06.342 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-15T21:06:06.342 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-15T21:06:06.342 INFO:teuthology.orchestra.run.smithi016.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-15T21:06:06.401 INFO:teuthology.orchestra.run.smithi016.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-15T21:06:06.401 INFO:teuthology.orchestra.run.smithi016.stdout:Need to get 244 MB of archives. 2024-09-15T21:06:06.401 INFO:teuthology.orchestra.run.smithi016.stdout:After this operation, 909 MB of additional disk space will be used. 2024-09-15T21:06:06.401 INFO:teuthology.orchestra.run.smithi016.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-15T21:06:06.483 INFO:teuthology.orchestra.run.smithi016.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4930-gfc3e0bff-1jammy [3,633 kB] 2024-09-15T21:06:06.488 INFO:teuthology.orchestra.run.smithi016.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-15T21:06:06.491 INFO:teuthology.orchestra.run.smithi016.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-15T21:06:06.508 INFO:teuthology.orchestra.run.smithi016.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-15T21:06:06.568 INFO:teuthology.orchestra.run.smithi016.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-15T21:06:06.572 INFO:teuthology.orchestra.run.smithi016.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-15T21:06:06.585 INFO:teuthology.orchestra.run.smithi016.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-15T21:06:06.589 INFO:teuthology.orchestra.run.smithi016.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-15T21:06:06.591 INFO:teuthology.orchestra.run.smithi016.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-15T21:06:06.591 INFO:teuthology.orchestra.run.smithi016.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-15T21:06:06.592 INFO:teuthology.orchestra.run.smithi016.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-15T21:06:06.601 INFO:teuthology.orchestra.run.smithi016.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-15T21:06:06.603 INFO:teuthology.orchestra.run.smithi016.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-15T21:06:06.605 INFO:teuthology.orchestra.run.smithi016.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-15T21:06:06.607 INFO:teuthology.orchestra.run.smithi016.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-15T21:06:06.622 INFO:teuthology.orchestra.run.smithi016.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-15T21:06:06.624 INFO:teuthology.orchestra.run.smithi016.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-15T21:06:06.627 INFO:teuthology.orchestra.run.smithi016.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-15T21:06:06.628 INFO:teuthology.orchestra.run.smithi016.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-15T21:06:06.629 INFO:teuthology.orchestra.run.smithi016.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-15T21:06:06.630 INFO:teuthology.orchestra.run.smithi016.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-15T21:06:06.639 INFO:teuthology.orchestra.run.smithi016.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-15T21:06:06.639 INFO:teuthology.orchestra.run.smithi016.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-15T21:06:06.640 INFO:teuthology.orchestra.run.smithi016.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-15T21:06:06.640 INFO:teuthology.orchestra.run.smithi016.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-15T21:06:06.656 INFO:teuthology.orchestra.run.smithi016.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-15T21:06:06.657 INFO:teuthology.orchestra.run.smithi016.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-15T21:06:06.657 INFO:teuthology.orchestra.run.smithi016.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-15T21:06:06.660 INFO:teuthology.orchestra.run.smithi016.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-15T21:06:06.660 INFO:teuthology.orchestra.run.smithi016.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-15T21:06:06.661 INFO:teuthology.orchestra.run.smithi016.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-15T21:06:06.673 INFO:teuthology.orchestra.run.smithi016.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-15T21:06:06.674 INFO:teuthology.orchestra.run.smithi016.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-15T21:06:06.674 INFO:teuthology.orchestra.run.smithi016.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-15T21:06:06.675 INFO:teuthology.orchestra.run.smithi016.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-15T21:06:06.690 INFO:teuthology.orchestra.run.smithi016.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-15T21:06:06.691 INFO:teuthology.orchestra.run.smithi016.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-15T21:06:06.691 INFO:teuthology.orchestra.run.smithi016.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-15T21:06:06.697 INFO:teuthology.orchestra.run.smithi016.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-15T21:06:06.697 INFO:teuthology.orchestra.run.smithi016.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-15T21:06:06.698 INFO:teuthology.orchestra.run.smithi016.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-15T21:06:06.708 INFO:teuthology.orchestra.run.smithi016.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-15T21:06:06.709 INFO:teuthology.orchestra.run.smithi016.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-15T21:06:06.711 INFO:teuthology.orchestra.run.smithi016.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-15T21:06:06.712 INFO:teuthology.orchestra.run.smithi016.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-15T21:06:06.725 INFO:teuthology.orchestra.run.smithi016.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-15T21:06:06.725 INFO:teuthology.orchestra.run.smithi016.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-15T21:06:06.728 INFO:teuthology.orchestra.run.smithi016.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-15T21:06:06.728 INFO:teuthology.orchestra.run.smithi016.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-15T21:06:06.729 INFO:teuthology.orchestra.run.smithi016.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-15T21:06:06.732 INFO:teuthology.orchestra.run.smithi016.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-15T21:06:06.774 INFO:teuthology.orchestra.run.smithi016.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4930-gfc3e0bff-1jammy [3,334 kB] 2024-09-15T21:06:06.785 INFO:teuthology.orchestra.run.smithi016.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-15T21:06:06.786 INFO:teuthology.orchestra.run.smithi016.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-15T21:06:06.787 INFO:teuthology.orchestra.run.smithi016.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-15T21:06:06.787 INFO:teuthology.orchestra.run.smithi016.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-15T21:06:06.789 INFO:teuthology.orchestra.run.smithi016.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-15T21:06:06.806 INFO:teuthology.orchestra.run.smithi016.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-15T21:06:06.808 INFO:teuthology.orchestra.run.smithi016.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-15T21:06:06.808 INFO:teuthology.orchestra.run.smithi016.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-15T21:06:06.823 INFO:teuthology.orchestra.run.smithi016.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-15T21:06:06.823 INFO:teuthology.orchestra.run.smithi016.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-15T21:06:06.823 INFO:teuthology.orchestra.run.smithi016.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-15T21:06:06.824 INFO:teuthology.orchestra.run.smithi016.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-15T21:06:06.824 INFO:teuthology.orchestra.run.smithi016.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-15T21:06:06.825 INFO:teuthology.orchestra.run.smithi016.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-15T21:06:06.840 INFO:teuthology.orchestra.run.smithi016.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-15T21:06:06.840 INFO:teuthology.orchestra.run.smithi016.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-15T21:06:06.843 INFO:teuthology.orchestra.run.smithi016.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-15T21:06:06.857 INFO:teuthology.orchestra.run.smithi016.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-15T21:06:06.858 INFO:teuthology.orchestra.run.smithi016.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-15T21:06:06.859 INFO:teuthology.orchestra.run.smithi016.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-15T21:06:06.862 INFO:teuthology.orchestra.run.smithi016.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-15T21:06:06.864 INFO:teuthology.orchestra.run.smithi016.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-15T21:06:06.865 INFO:teuthology.orchestra.run.smithi016.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-15T21:06:06.865 INFO:teuthology.orchestra.run.smithi016.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-15T21:06:06.874 INFO:teuthology.orchestra.run.smithi016.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-15T21:06:06.874 INFO:teuthology.orchestra.run.smithi016.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-15T21:06:06.877 INFO:teuthology.orchestra.run.smithi016.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-15T21:06:06.891 INFO:teuthology.orchestra.run.smithi016.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-15T21:06:06.892 INFO:teuthology.orchestra.run.smithi016.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-15T21:06:06.892 INFO:teuthology.orchestra.run.smithi016.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-15T21:06:06.893 INFO:teuthology.orchestra.run.smithi016.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-15T21:06:06.893 INFO:teuthology.orchestra.run.smithi016.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-15T21:06:06.899 INFO:teuthology.orchestra.run.smithi016.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4930-gfc3e0bff-1jammy [721 kB] 2024-09-15T21:06:06.899 INFO:teuthology.orchestra.run.smithi016.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-15T21:06:06.900 INFO:teuthology.orchestra.run.smithi016.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-15T21:06:06.908 INFO:teuthology.orchestra.run.smithi016.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-15T21:06:06.909 INFO:teuthology.orchestra.run.smithi016.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-15T21:06:06.910 INFO:teuthology.orchestra.run.smithi016.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-15T21:06:06.921 INFO:teuthology.orchestra.run.smithi016.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4930-gfc3e0bff-1jammy [356 kB] 2024-09-15T21:06:06.928 INFO:teuthology.orchestra.run.smithi016.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4930-gfc3e0bff-1jammy [32.8 kB] 2024-09-15T21:06:06.930 INFO:teuthology.orchestra.run.smithi016.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4930-gfc3e0bff-1jammy [180 kB] 2024-09-15T21:06:06.939 INFO:teuthology.orchestra.run.smithi016.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4930-gfc3e0bff-1jammy [71.3 kB] 2024-09-15T21:06:06.941 INFO:teuthology.orchestra.run.smithi016.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4930-gfc3e0bff-1jammy [338 kB] 2024-09-15T21:06:06.948 INFO:teuthology.orchestra.run.smithi016.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4930-gfc3e0bff-1jammy [7,966 kB] 2024-09-15T21:06:07.137 INFO:teuthology.orchestra.run.smithi016.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4930-gfc3e0bff-1jammy [112 kB] 2024-09-15T21:06:07.139 INFO:teuthology.orchestra.run.smithi016.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4930-gfc3e0bff-1jammy [473 kB] 2024-09-15T21:06:07.147 INFO:teuthology.orchestra.run.smithi016.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4930-gfc3e0bff-1jammy [28.2 MB] 2024-09-15T21:06:07.709 INFO:teuthology.orchestra.run.smithi016.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4930-gfc3e0bff-1jammy [5,206 kB] 2024-09-15T21:06:07.782 INFO:teuthology.orchestra.run.smithi016.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4930-gfc3e0bff-1jammy [251 kB] 2024-09-15T21:06:07.785 INFO:teuthology.orchestra.run.smithi016.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4930-gfc3e0bff-1jammy [124 kB] 2024-09-15T21:06:07.786 INFO:teuthology.orchestra.run.smithi016.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4930-gfc3e0bff-1jammy [1,561 kB] 2024-09-15T21:06:07.800 INFO:teuthology.orchestra.run.smithi016.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4930-gfc3e0bff-1jammy [6,309 kB] 2024-09-15T21:06:07.889 INFO:teuthology.orchestra.run.smithi016.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4930-gfc3e0bff-1jammy [23.1 MB] 2024-09-15T21:06:08.365 INFO:teuthology.orchestra.run.smithi016.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4930-gfc3e0bff-1jammy [14.1 kB] 2024-09-15T21:06:08.366 INFO:teuthology.orchestra.run.smithi016.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4930-gfc3e0bff-1jammy [880 kB] 2024-09-15T21:06:08.376 INFO:teuthology.orchestra.run.smithi016.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4930-gfc3e0bff-1jammy [2,452 kB] 2024-09-15T21:06:08.405 INFO:teuthology.orchestra.run.smithi016.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4930-gfc3e0bff-1jammy [813 kB] 2024-09-15T21:06:08.414 INFO:teuthology.orchestra.run.smithi016.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4930-gfc3e0bff-1jammy [160 kB] 2024-09-15T21:06:08.416 INFO:teuthology.orchestra.run.smithi016.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4930-gfc3e0bff-1jammy [61.4 MB] 2024-09-15T21:06:09.710 INFO:teuthology.orchestra.run.smithi016.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4930-gfc3e0bff-1jammy [8,625 kB] 2024-09-15T21:06:09.834 INFO:teuthology.orchestra.run.smithi016.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4930-gfc3e0bff-1jammy [14.2 kB] 2024-09-15T21:06:09.834 INFO:teuthology.orchestra.run.smithi016.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4930-gfc3e0bff-1jammy [48.5 MB] 2024-09-15T21:06:10.833 INFO:teuthology.orchestra.run.smithi016.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4930-gfc3e0bff-1jammy [130 kB] 2024-09-15T21:06:10.836 INFO:teuthology.orchestra.run.smithi016.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4930-gfc3e0bff-1jammy [39.0 kB] 2024-09-15T21:06:10.837 INFO:teuthology.orchestra.run.smithi016.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4930-gfc3e0bff-1jammy [15.8 MB] 2024-09-15T21:06:11.138 INFO:teuthology.orchestra.run.smithi016.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4930-gfc3e0bff-1jammy [91.2 kB] 2024-09-15T21:06:11.733 INFO:teuthology.orchestra.run.smithi016.stdout:Fetched 244 MB in 5s (50.9 MB/s) 2024-09-15T21:06:11.895 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-15T21:06:11.998 INFO:teuthology.orchestra.run.smithi016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-15T21:06:12.003 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-15T21:06:12.076 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-15T21:06:12.355 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-15T21:06:12.382 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-15T21:06:12.413 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-15T21:06:12.858 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-15T21:06:12.885 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-15T21:06:12.933 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-15T21:06:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-15T21:06:13.397 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T21:06:13.445 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:06:13.840 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-15T21:06:13.867 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T21:06:13.907 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:06:14.227 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-15T21:06:14.253 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T21:06:14.293 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:06:14.630 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-15T21:06:14.657 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-15T21:06:14.688 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-15T21:06:14.991 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package librados2. 2024-09-15T21:06:15.018 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../007-librados2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:15.057 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking librados2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:15.577 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libnbd0. 2024-09-15T21:06:15.604 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-15T21:06:15.644 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-15T21:06:15.955 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package librbd1. 2024-09-15T21:06:15.982 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../009-librbd1_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:16.022 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking librbd1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:16.442 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libcephfs2. 2024-09-15T21:06:16.469 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:16.508 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libcephfs2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:16.794 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-rados. 2024-09-15T21:06:16.821 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:16.860 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-rados (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:17.189 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-15T21:06:17.215 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:06:17.255 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-ceph-argparse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:17.524 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-cephfs. 2024-09-15T21:06:17.552 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:17.591 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-cephfs (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:17.869 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-15T21:06:17.885 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:06:17.927 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:18.257 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-15T21:06:18.284 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-15T21:06:18.324 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-15T21:06:18.635 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-prettytable. 2024-09-15T21:06:18.662 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-15T21:06:18.701 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-15T21:06:18.929 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-rbd. 2024-09-15T21:06:18.945 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:18.979 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-rbd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:19.307 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-15T21:06:19.334 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-15T21:06:19.373 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-15T21:06:19.651 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-15T21:06:19.678 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-15T21:06:19.717 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-15T21:06:20.082 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-15T21:06:20.109 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-15T21:06:20.345 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-15T21:06:20.649 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package lua5.1. 2024-09-15T21:06:20.676 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-15T21:06:20.715 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-15T21:06:21.027 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package lua-any. 2024-09-15T21:06:21.043 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-15T21:06:21.085 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-15T21:06:21.329 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package zip. 2024-09-15T21:06:21.345 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-15T21:06:21.387 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking zip (3.0-12build2) ... 2024-09-15T21:06:21.674 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package unzip. 2024-09-15T21:06:21.701 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-15T21:06:21.732 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-15T21:06:22.161 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package luarocks. 2024-09-15T21:06:22.188 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-15T21:06:22.228 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-15T21:06:22.586 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package librgw2. 2024-09-15T21:06:22.602 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../026-librgw2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:22.644 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking librgw2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:23.263 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-rgw. 2024-09-15T21:06:23.290 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:23.329 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-rgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:23.632 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-15T21:06:23.648 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-15T21:06:23.682 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-15T21:06:23.977 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libradosstriper1. 2024-09-15T21:06:24.004 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libradosstriper1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:24.514 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-common. 2024-09-15T21:06:24.540 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:24.580 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:26.219 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-base. 2024-09-15T21:06:26.246 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:26.287 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-base (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:26.865 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-15T21:06:26.892 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-15T21:06:26.931 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-15T21:06:27.243 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-cheroot. 2024-09-15T21:06:27.270 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-15T21:06:27.309 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-15T21:06:27.646 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-15T21:06:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-15T21:06:27.712 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-15T21:06:27.973 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-15T21:06:27.989 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-15T21:06:28.023 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-15T21:06:28.293 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-15T21:06:28.320 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-15T21:06:28.359 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-15T21:06:28.671 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-tempora. 2024-09-15T21:06:28.697 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-15T21:06:28.737 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-15T21:06:29.057 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-portend. 2024-09-15T21:06:29.084 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-15T21:06:29.123 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-15T21:06:29.410 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-15T21:06:29.436 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-15T21:06:29.476 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-15T21:06:29.754 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-15T21:06:29.781 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-15T21:06:29.820 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-15T21:06:30.166 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-natsort. 2024-09-15T21:06:30.193 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-15T21:06:30.232 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-15T21:06:30.802 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-logutils. 2024-09-15T21:06:30.818 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-15T21:06:30.852 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-15T21:06:31.121 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-15T21:06:31.149 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-15T21:06:31.188 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-15T21:06:31.525 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-mako. 2024-09-15T21:06:31.552 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-15T21:06:31.591 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-15T21:06:31.919 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-15T21:06:31.946 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-15T21:06:31.986 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-15T21:06:32.263 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-15T21:06:32.279 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-15T21:06:32.313 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-15T21:06:32.575 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-webob. 2024-09-15T21:06:32.602 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-15T21:06:32.641 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-15T21:06:32.969 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-waitress. 2024-09-15T21:06:32.996 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-15T21:06:33.039 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-15T21:06:33.347 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-tempita. 2024-09-15T21:06:33.374 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-15T21:06:33.413 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-15T21:06:33.750 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-paste. 2024-09-15T21:06:33.777 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-15T21:06:33.817 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-15T21:06:34.120 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-15T21:06:34.136 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-15T21:06:34.178 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-15T21:06:34.481 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-15T21:06:34.508 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-15T21:06:34.547 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-15T21:06:34.876 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-webtest. 2024-09-15T21:06:34.903 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-15T21:06:34.934 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-15T21:06:35.187 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pecan. 2024-09-15T21:06:35.214 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-15T21:06:35.253 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-15T21:06:35.575 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-certifi. 2024-09-15T21:06:35.602 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-15T21:06:35.641 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-15T21:06:35.994 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-idna. 2024-09-15T21:06:36.021 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-15T21:06:36.061 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-15T21:06:36.380 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-urllib3. 2024-09-15T21:06:36.407 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-15T21:06:36.448 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-15T21:06:36.784 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-requests. 2024-09-15T21:06:36.811 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-15T21:06:36.850 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-15T21:06:37.136 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-15T21:06:37.161 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-15T21:06:37.194 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-15T21:06:37.481 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-15T21:06:37.508 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:06:37.581 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:37.978 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-15T21:06:38.005 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:38.380 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-15T21:06:38.407 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-15T21:06:38.447 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-15T21:06:38.749 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-openssl. 2024-09-15T21:06:38.777 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-15T21:06:38.816 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-15T21:06:39.077 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr. 2024-09-15T21:06:39.093 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:39.136 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:39.531 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mon. 2024-09-15T21:06:39.558 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:39.597 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mon (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:41.352 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-osd. 2024-09-15T21:06:41.378 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:41.409 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-osd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:42.554 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph. 2024-09-15T21:06:42.570 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../067-ceph_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:42.612 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:42.940 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-fuse. 2024-09-15T21:06:42.967 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:43.006 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:43.376 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mds. 2024-09-15T21:06:43.404 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:43.443 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mds (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:43.779 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package cephadm. 2024-09-15T21:06:43.795 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../070-cephadm_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:43.829 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:44.157 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-15T21:06:44.183 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-15T21:06:44.223 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-15T21:06:44.526 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-15T21:06:44.553 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-15T21:06:44.584 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-15T21:06:45.598 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-babel. 2024-09-15T21:06:45.625 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-15T21:06:45.664 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-15T21:06:45.984 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jinja2. 2024-09-15T21:06:46.011 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-15T21:06:46.050 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-15T21:06:46.412 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-15T21:06:46.439 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:06:46.478 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:46.807 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-15T21:06:46.834 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-15T21:06:46.873 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-15T21:06:47.184 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-routes. 2024-09-15T21:06:47.212 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-15T21:06:47.251 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-15T21:06:47.613 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-15T21:06:47.640 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:06:47.679 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:50.417 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-15T21:06:50.435 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-15T21:06:50.475 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-15T21:06:50.979 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-joblib. 2024-09-15T21:06:51.007 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-15T21:06:51.046 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-15T21:06:51.608 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-15T21:06:51.636 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-15T21:06:51.674 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-15T21:06:51.977 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-sklearn. 2024-09-15T21:06:52.005 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-15T21:06:52.145 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-15T21:06:52.814 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-15T21:06:52.842 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:06:52.880 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:53.841 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-cachetools. 2024-09-15T21:06:53.869 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-15T21:06:53.907 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-15T21:06:54.169 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-15T21:06:54.197 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-15T21:06:54.236 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-15T21:06:54.581 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-15T21:06:54.609 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-15T21:06:54.647 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-15T21:06:54.958 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-rsa. 2024-09-15T21:06:54.987 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-15T21:06:55.025 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-15T21:06:55.346 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-google-auth. 2024-09-15T21:06:55.374 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-15T21:06:55.412 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-15T21:06:55.732 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-15T21:06:55.760 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-15T21:06:55.799 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-15T21:06:56.127 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-websocket. 2024-09-15T21:06:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-15T21:06:56.193 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-15T21:06:56.508 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-15T21:06:56.533 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-15T21:06:56.563 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-15T21:06:57.261 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-15T21:06:57.289 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:06:57.328 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:06:57.655 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-15T21:06:57.683 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-15T21:06:57.721 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-15T21:06:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-15T21:06:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-15T21:06:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-15T21:06:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package jq. 2024-09-15T21:06:58.337 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-15T21:06:58.379 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-15T21:06:58.632 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package socat. 2024-09-15T21:06:58.660 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-15T21:06:58.698 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-15T21:06:58.993 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package xmlstarlet. 2024-09-15T21:06:59.010 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-15T21:06:59.043 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-15T21:06:59.381 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-test. 2024-09-15T21:06:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:06:59.439 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-test (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:01.844 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-volume. 2024-09-15T21:07:01.872 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:07:01.910 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-volume (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:02.180 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-15T21:07:02.198 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:07:02.230 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libcephfs-dev (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:02.508 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-15T21:07:02.536 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-15T21:07:02.566 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-15T21:07:02.904 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-15T21:07:02.933 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-15T21:07:02.970 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-15T21:07:03.333 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package nvme-cli. 2024-09-15T21:07:03.361 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-15T21:07:03.399 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-15T21:07:03.847 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package pkg-config. 2024-09-15T21:07:03.875 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-15T21:07:03.913 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-15T21:07:04.149 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-15T21:07:04.166 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-15T21:07:04.199 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-15T21:07:04.585 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-15T21:07:04.614 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-15T21:07:04.643 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-15T21:07:04.922 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pastescript. 2024-09-15T21:07:04.950 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-15T21:07:04.988 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-15T21:07:05.308 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pluggy. 2024-09-15T21:07:05.337 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-15T21:07:05.374 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-15T21:07:05.669 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-psutil. 2024-09-15T21:07:05.697 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-15T21:07:05.735 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-15T21:07:06.080 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-py. 2024-09-15T21:07:06.106 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-15T21:07:06.147 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-15T21:07:06.492 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pygments. 2024-09-15T21:07:06.521 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-15T21:07:06.558 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-15T21:07:07.355 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-15T21:07:07.384 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-15T21:07:07.421 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-15T21:07:07.674 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-toml. 2024-09-15T21:07:07.702 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-15T21:07:07.740 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-15T21:07:08.043 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pytest. 2024-09-15T21:07:08.072 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-15T21:07:08.109 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-15T21:07:08.447 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-simplejson. 2024-09-15T21:07:08.475 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-15T21:07:08.513 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-15T21:07:08.757 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-15T21:07:08.774 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-15T21:07:08.815 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-15T21:07:09.545 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package radosgw. 2024-09-15T21:07:09.574 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../117-radosgw_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:07:09.611 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking radosgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:10.556 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package rbd-fuse. 2024-09-15T21:07:10.573 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:07:10.614 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking rbd-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:10.911 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-15T21:07:11.236 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-15T21:07:11.361 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-15T21:07:11.647 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-15T21:07:11.995 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-15T21:07:12.434 INFO:teuthology.orchestra.run.smithi016.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-15T21:07:12.664 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:13.131 INFO:teuthology.orchestra.run.smithi016.stdout:Adding system user cephadm....done 2024-09-15T21:07:13.304 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-15T21:07:13.613 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-15T21:07:13.873 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-15T21:07:13.974 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-15T21:07:14.217 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-15T21:07:14.509 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-15T21:07:14.635 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-15T21:07:14.954 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-15T21:07:15.348 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-15T21:07:15.608 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-15T21:07:15.925 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-15T21:07:16.194 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-15T21:07:16.469 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-15T21:07:16.779 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-15T21:07:17.080 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-ceph-argparse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:17.381 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-15T21:07:17.549 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-15T21:07:17.700 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-15T21:07:17.851 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-15T21:07:18.011 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-15T21:07:18.295 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-15T21:07:18.666 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-15T21:07:18.701 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-15T21:07:18.765 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-15T21:07:18.874 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-15T21:07:19.259 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-15T21:07:19.410 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-15T21:07:19.712 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-15T21:07:19.987 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-15T21:07:20.314 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up zip (3.0-12build2) ... 2024-09-15T21:07:20.439 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-15T21:07:21.133 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-15T21:07:21.409 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-15T21:07:21.535 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-15T21:07:21.661 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-15T21:07:21.971 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-15T21:07:22.397 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-15T21:07:22.840 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-15T21:07:23.275 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-15T21:07:23.593 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-15T21:07:23.961 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-15T21:07:24.245 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-15T21:07:24.405 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:24.740 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-15T21:07:25.049 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-15T21:07:26.297 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-15T21:07:26.557 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:07:26.699 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-15T21:07:26.950 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-15T21:07:27.076 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-15T21:07:27.202 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-15T21:07:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-15T21:07:27.805 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-15T21:07:28.056 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:07:28.165 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-15T21:07:28.450 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-15T21:07:28.709 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-15T21:07:28.977 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-15T21:07:29.245 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-15T21:07:29.504 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-15T21:07:29.772 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-15T21:07:29.890 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-15T21:07:30.157 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-15T21:07:30.275 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-15T21:07:30.585 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-15T21:07:30.886 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-15T21:07:31.145 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-15T21:07:31.296 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-15T21:07:31.414 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-15T21:07:31.486 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-15T21:07:31.715 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-15T21:07:31.826 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-15T21:07:32.218 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-15T21:07:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-15T21:07:32.603 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-15T21:07:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:07:32.997 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-15T21:07:33.307 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-15T21:07:33.599 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-15T21:07:33.876 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-15T21:07:34.002 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-15T21:07:34.278 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-15T21:07:34.631 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-15T21:07:34.882 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-15T21:07:35.259 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-15T21:07:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-15T21:07:35.896 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-15T21:07:36.023 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up librados2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:36.132 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:36.233 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-15T21:07:36.577 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libcephfs2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:36.703 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libradosstriper1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:36.838 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-15T21:07:37.123 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up librbd1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:37.224 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-modules-core (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:37.342 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:37.566 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T21:07:37.566 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T21:07:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libcephfs-dev (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:38.186 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-rados (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:38.312 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up librgw2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:38.446 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-15T21:07:39.691 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-rbd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:39.817 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up rbd-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:39.943 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-rgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:40.069 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-cephfs (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:40.195 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:40.739 INFO:teuthology.orchestra.run.smithi016.stdout:Adding group ceph....done 2024-09-15T21:07:41.107 INFO:teuthology.orchestra.run.smithi016.stdout:Adding system user ceph....done 2024-09-15T21:07:41.275 INFO:teuthology.orchestra.run.smithi016.stdout:Setting system user ceph properties....done 2024-09-15T21:07:41.289 INFO:teuthology.orchestra.run.smithi016.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-15T21:07:41.444 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-15T21:07:41.780 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-15T21:07:42.280 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-test (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:42.432 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up radosgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:07:43.545 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-base (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:44.020 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-15T21:07:44.503 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mds (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:44.732 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T21:07:44.732 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T21:07:45.214 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:45.431 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T21:07:45.431 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T21:07:45.915 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-osd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:46.184 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T21:07:46.184 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T21:07:46.641 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:46.742 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:46.885 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mon (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:47.104 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T21:07:47.104 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T21:07:47.570 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:47.721 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:47.849 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-dashboard (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:47.975 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-volume (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:07:48.722 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-15T21:07:48.817 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-15T21:07:48.952 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-15T21:07:49.744 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-15T21:07:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T21:07:50.387 DEBUG:teuthology.parallel:result is None 2024-09-15T21:07:50.387 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:07:50.530 DEBUG:teuthology.orchestra.run.smithi016:> dpkg-query -W -f '${Version}' ceph 2024-09-15T21:07:50.550 INFO:teuthology.orchestra.run.smithi016.stdout:19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:07:50.550 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:07:50.550 INFO:teuthology.task.install:The correct ceph version 19.3.0-4930-gfc3e0bff-1jammy is installed. 2024-09-15T21:07:50.552 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T21:07:50.552 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T21:07:50.552 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T21:07:50.604 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T21:07:50.604 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T21:07:50.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T21:07:50.659 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T21:07:50.715 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T21:07:50.715 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T21:07:50.715 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T21:07:50.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T21:07:50.827 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T21:07:50.827 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T21:07:50.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T21:07:50.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T21:07:50.939 INFO:teuthology.run_tasks:Running task workunit... 2024-09-15T21:07:50.955 INFO:tasks.workunit:Pulling workunits from ref 7498493279982dfe87f00616198a5967475f6169 2024-09-15T21:07:50.955 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-15T21:07:50.955 INFO:tasks.workunit:timeout=3h 2024-09-15T21:07:50.956 INFO:tasks.workunit:cleanup=True 2024-09-15T21:07:50.956 DEBUG:teuthology.orchestra.run.smithi016:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-15T21:07:50.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:07:50.988 INFO:teuthology.orchestra.run.smithi016.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-15T21:07:50.989 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-15T21:07:51.036 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-15T21:07:51.036 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-15T21:07:51.084 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7498493279982dfe87f00616198a5967475f6169 2024-09-15T21:07:51.134 INFO:tasks.workunit.client.0.smithi016.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-15T21:08:44.921 INFO:tasks.workunit.client.0.smithi016.stderr:Updating files: 93% (12098/12889) Updating files: 94% (12116/12889) Updating files: 95% (12245/12889) Updating files: 96% (12374/12889) Updating files: 97% (12503/12889) Updating files: 98% (12632/12889) Updating files: 99% (12761/12889) Updating files: 100% (12889/12889) Updating files: 100% (12889/12889), done. 2024-09-15T21:08:45.575 INFO:tasks.workunit.client.0.smithi016.stderr:Note: switching to '7498493279982dfe87f00616198a5967475f6169'. 2024-09-15T21:08:45.575 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-15T21:08:45.575 INFO:tasks.workunit.client.0.smithi016.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-15T21:08:45.575 INFO:tasks.workunit.client.0.smithi016.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-15T21:08:45.576 INFO:tasks.workunit.client.0.smithi016.stderr:state without impacting any branches by switching back to a branch. 2024-09-15T21:08:45.576 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-15T21:08:45.576 INFO:tasks.workunit.client.0.smithi016.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-15T21:08:45.576 INFO:tasks.workunit.client.0.smithi016.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-15T21:08:45.576 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-15T21:08:45.576 INFO:tasks.workunit.client.0.smithi016.stderr: git switch -c 2024-09-15T21:08:45.576 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-15T21:08:45.576 INFO:tasks.workunit.client.0.smithi016.stderr:Or undo this operation with: 2024-09-15T21:08:45.576 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-15T21:08:45.576 INFO:tasks.workunit.client.0.smithi016.stderr: git switch - 2024-09-15T21:08:45.576 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-15T21:08:45.576 INFO:tasks.workunit.client.0.smithi016.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-15T21:08:45.577 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-15T21:08:45.577 INFO:tasks.workunit.client.0.smithi016.stderr:HEAD is now at 74984932799 Merge pull request #59797 from zdover23/wip-doc-2024-09-15-install-install-storage-cluster 2024-09-15T21:08:45.578 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-15T21:08:45.625 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T21:08:45.625 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-15T21:08:45.672 INFO:tasks.workunit:Running workunits matching crush on client.0... 2024-09-15T21:08:45.672 INFO:tasks.workunit:Running workunit crush/crush-choose-args.sh... 2024-09-15T21:08:45.673 DEBUG:teuthology.orchestra.run.smithi016:workunit test crush/crush-choose-args.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7498493279982dfe87f00616198a5967475f6169 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh 2024-09-15T21:08:45.784 INFO:tasks.workunit.client.0.smithi016.stderr:stty: 'standard input': Inappropriate ioctl for device 2024-09-15T21:08:45.789 INFO:tasks.workunit.client.0.smithi016.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2024-09-15T21:08:45.789 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2315: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-09-15T21:08:45.789 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2315: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-09-15T21:08:45.790 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2316: main: export PYTHONWARNINGS=ignore 2024-09-15T21:08:45.790 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2316: main: PYTHONWARNINGS=ignore 2024-09-15T21:08:45.790 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2317: main: export CEPH_CONF=/dev/null 2024-09-15T21:08:45.790 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2317: main: CEPH_CONF=/dev/null 2024-09-15T21:08:45.790 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2318: main: unset CEPH_ARGS 2024-09-15T21:08:45.791 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2320: main: local code 2024-09-15T21:08:45.791 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2321: main: run td/crush-choose-args 2024-09-15T21:08:45.791 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:21: run: local dir=td/crush-choose-args 2024-09-15T21:08:45.791 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:22: run: shift 2024-09-15T21:08:45.791 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:24: run: export CEPH_MON=127.0.0.1:7131 2024-09-15T21:08:45.791 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:24: run: CEPH_MON=127.0.0.1:7131 2024-09-15T21:08:45.792 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:25: run: export CEPH_ARGS 2024-09-15T21:08:45.792 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:26: run: uuidgen 2024-09-15T21:08:45.793 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:26: run: CEPH_ARGS+='--fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none ' 2024-09-15T21:08:45.793 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:27: run: CEPH_ARGS+='--mon-host=127.0.0.1:7131 ' 2024-09-15T21:08:45.793 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:28: run: CEPH_ARGS+='--crush-location=root=default,host=HOST ' 2024-09-15T21:08:45.794 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:29: run: CEPH_ARGS+='--osd-crush-initial-weight=3 ' 2024-09-15T21:08:45.794 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:35: run: CEPH_ARGS+='--osd-class-update-on-start=false ' 2024-09-15T21:08:45.794 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:37: run: set 2024-09-15T21:08:45.794 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:37: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2024-09-15T21:08:45.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:37: run: local 'funcs=TEST_choose_args_update 2024-09-15T21:08:45.797 INFO:tasks.workunit.client.0.smithi016.stderr:TEST_move_bucket 2024-09-15T21:08:45.797 INFO:tasks.workunit.client.0.smithi016.stderr:TEST_no_update_weight_set 2024-09-15T21:08:45.797 INFO:tasks.workunit.client.0.smithi016.stderr:TEST_reweight' 2024-09-15T21:08:45.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:38: run: for func in $funcs 2024-09-15T21:08:45.798 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:39: run: setup td/crush-choose-args 2024-09-15T21:08:45.798 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/crush-choose-args 2024-09-15T21:08:45.798 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/crush-choose-args 2024-09-15T21:08:45.798 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-choose-args 2024-09-15T21:08:45.798 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:08:45.798 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-choose-args KILL 2024-09-15T21:08:45.799 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:08:45.799 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:08:45.799 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:08:45.799 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:08:45.799 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:08:45.801 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:08:45.801 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:08:45.801 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:08:45.802 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:08:45.802 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:08:45.802 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:08:45.802 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:08:45.804 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:08:45.804 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:08:45.804 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:08:45.804 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:08:45.804 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:08:45.805 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:08:45.805 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-choose-args 2024-09-15T21:08:45.806 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:08:45.807 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:08:45.807 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:08:45.807 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.21400 2024-09-15T21:08:45.808 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:08:45.808 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:08:45.808 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/crush-choose-args 2024-09-15T21:08:45.809 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2024-09-15T21:08:45.809 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:08:45.810 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:08:45.810 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.21400 2024-09-15T21:08:45.813 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2024-09-15T21:08:45.813 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2024-09-15T21:08:45.813 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2024-09-15T21:08:45.813 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2024-09-15T21:08:45.813 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/crush-choose-args 1' TERM HUP INT 2024-09-15T21:08:45.814 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:40: run: TEST_choose_args_update td/crush-choose-args 2024-09-15T21:08:45.814 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:49: TEST_choose_args_update: local dir=td/crush-choose-args 2024-09-15T21:08:45.814 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:51: TEST_choose_args_update: run_mon td/crush-choose-args a 2024-09-15T21:08:45.814 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/crush-choose-args 2024-09-15T21:08:45.814 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2024-09-15T21:08:45.815 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2024-09-15T21:08:45.815 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2024-09-15T21:08:45.815 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/crush-choose-args/a 2024-09-15T21:08:45.815 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/crush-choose-args/a --run-dir=td/crush-choose-args 2024-09-15T21:08:46.419 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2024-09-15T21:08:46.419 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:08:46.419 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:08:46.420 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:08:46.420 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:08:46.420 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:08:46.420 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:08:46.420 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/crush-choose-args/a '--log-file=td/crush-choose-args/$name.log' '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --mon-cluster-log-file=td/crush-choose-args/log --run-dir=td/crush-choose-args '--pid-file=td/crush-choose-args/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2024-09-15T21:08:46.693 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2024-09-15T21:08:46.694 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2024-09-15T21:08:46.694 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:08:46.694 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:08:46.694 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2024-09-15T21:08:46.695 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2024-09-15T21:08:46.695 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:08:46.695 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:08:46.695 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:08:46.696 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:08:46.696 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:08:46.696 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:08:46.696 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.21400/ceph-mon.a.asok 2024-09-15T21:08:46.696 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:08:46.696 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.21400/ceph-mon.a.asok config get fsid 2024-09-15T21:08:46.813 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2024-09-15T21:08:46.813 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:08:46.813 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:08:46.813 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2024-09-15T21:08:46.814 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2024-09-15T21:08:46.814 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:08:46.814 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:08:46.814 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:08:46.814 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:08:46.814 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:08:46.814 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:08:46.814 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.21400/ceph-mon.a.asok 2024-09-15T21:08:46.815 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:08:46.815 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.21400/ceph-mon.a.asok config get mon_host 2024-09-15T21:08:46.920 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:52: TEST_choose_args_update: run_mgr td/crush-choose-args x 2024-09-15T21:08:46.920 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/crush-choose-args 2024-09-15T21:08:46.920 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2024-09-15T21:08:46.920 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2024-09-15T21:08:46.920 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2024-09-15T21:08:46.920 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/crush-choose-args/x 2024-09-15T21:08:46.920 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2024-09-15T21:08:47.835 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2024-09-15T21:08:47.836 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:08:47.836 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:08:47.836 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:08:47.836 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:08:47.836 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:08:47.836 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:08:47.837 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2024-09-15T21:08:47.971 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/crush-choose-args/x '--log-file=td/crush-choose-args/$name.log' '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --run-dir=td/crush-choose-args '--pid-file=td/crush-choose-args/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2024-09-15T21:08:48.098 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:53: TEST_choose_args_update: run_osd td/crush-choose-args 0 2024-09-15T21:08:48.099 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-choose-args 2024-09-15T21:08:48.099 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:08:48.099 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2024-09-15T21:08:48.099 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:08:48.099 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-choose-args/0 2024-09-15T21:08:48.099 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false ' 2024-09-15T21:08:48.099 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:08:48.099 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:08:48.099 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:08:48.099 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-choose-args/0' 2024-09-15T21:08:48.099 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-choose-args/0/journal' 2024-09-15T21:08:48.100 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:08:48.100 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:08:48.100 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-choose-args' 2024-09-15T21:08:48.100 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:08:48.100 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:08:48.100 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:08:48.101 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:08:48.101 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:08:48.101 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:08:48.101 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:08:48.101 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:08:48.101 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:08:48.101 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:08:48.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:08:48.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-choose-args/$name.log' 2024-09-15T21:08:48.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-choose-args/$name.pid' 2024-09-15T21:08:48.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:08:48.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:08:48.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:08:48.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:08:48.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:08:48.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:08:48.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-choose-args/0 2024-09-15T21:08:48.103 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:08:48.103 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=31048323-58bd-444d-882e-964d1933676a 2024-09-15T21:08:48.103 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 31048323-58bd-444d-882e-964d1933676a' 2024-09-15T21:08:48.103 INFO:tasks.workunit.client.0.smithi016.stdout:add osd0 31048323-58bd-444d-882e-964d1933676a 2024-09-15T21:08:48.103 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:08:48.125 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDgTOdm+JBOBxAANC2tMz8vRRHOHpo3iSG8dg== 2024-09-15T21:08:48.125 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDgTOdm+JBOBxAANC2tMz8vRRHOHpo3iSG8dg=="}' 2024-09-15T21:08:48.125 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 31048323-58bd-444d-882e-964d1933676a -i td/crush-choose-args/0/new.json 2024-09-15T21:08:48.423 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:08:48.439 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-choose-args/0/new.json 2024-09-15T21:08:48.439 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/0 --osd-journal=td/crush-choose-args/0/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDgTOdm+JBOBxAANC2tMz8vRRHOHpo3iSG8dg== --osd-uuid 31048323-58bd-444d-882e-964d1933676a 2024-09-15T21:08:48.528 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:08:48.523+0000 7f5f0512a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:08:48.530 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:08:48.523+0000 7f5f0512a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:08:48.531 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:08:48.527+0000 7f5f0512a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:08:48.531 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:08:48.527+0000 7f5f0512a7c0 -1 bdev(0x55f44a3b9c00 td/crush-choose-args/0/block) open stat got: (1) Operation not permitted 2024-09-15T21:08:48.531 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:08:48.527+0000 7f5f0512a7c0 -1 bluestore(td/crush-choose-args/0) _read_fsid unparsable uuid 2024-09-15T21:08:54.652 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-choose-args/0/keyring 2024-09-15T21:08:54.652 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:08:54.653 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd0 key to auth repository 2024-09-15T21:08:54.653 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2024-09-15T21:08:54.653 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-choose-args/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:08:54.908 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.0 2024-09-15T21:08:54.908 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2024-09-15T21:08:54.908 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/0 --osd-journal=td/crush-choose-args/0/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:08:54.909 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:08:54.909 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:08:54.909 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:08:54.944 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:08:54.939+0000 7fc3ef6d87c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:08:54.946 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:08:54.939+0000 7fc3ef6d87c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:08:54.947 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:08:54.943+0000 7fc3ef6d87c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:08:55.132 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2024-09-15T21:08:55.133 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:08:55.133 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2024-09-15T21:08:55.133 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:08:55.133 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:08:55.133 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:08:55.133 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:08:55.134 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:08:55.134 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:08:55.134 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:08:55.380 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:08:56.032 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:08:56.027+0000 7fc3ef6d87c0 -1 Falling back to public interface 2024-09-15T21:08:56.382 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:08:56.382 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:08:56.382 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:08:56.382 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:08:56.383 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:08:56.383 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:08:56.616 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:08:57.617 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:08:57.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:08:57.618 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:08:57.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:08:57.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:08:57.619 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:08:58.114 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:08:58.683 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:08:58.679+0000 7fc3ef6d87c0 -1 osd.0 0 log_to_monitors true 2024-09-15T21:08:59.116 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:08:59.116 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:08:59.116 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:08:59.117 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:08:59.117 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:08:59.117 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:08:59.678 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:00.680 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:00.680 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:00.681 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:09:00.681 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:09:00.681 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:00.681 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:01.352 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:02.354 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:02.354 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:02.354 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:09:02.355 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:09:02.355 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:02.355 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:02.811 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:03.813 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:03.813 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:03.813 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:09:03.813 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:09:03.814 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:03.814 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:04.377 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:05.378 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:05.378 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:05.378 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:09:05.379 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:09:05.379 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:05.379 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:05.821 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:06.823 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:06.823 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:06.823 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:09:06.824 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:09:06.824 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:06.824 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:07.387 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:08.389 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:08.389 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:08.389 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:09:08.389 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:09:08.390 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:08.390 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:08.879 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:09.880 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:09.881 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:09:09.881 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:09.882 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:09:09.882 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:09.882 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:10.589 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:11.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:11.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:11.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:09:11.592 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:09:11.592 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:11.592 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:12.227 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:13.229 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:13.230 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:13.230 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:09:13.230 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:09:13.230 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:13.231 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:13.801 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:14.803 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:14.803 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:14.803 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:09:14.803 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:09:14.804 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:14.804 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:16.368 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:17.370 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:17.370 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:17.370 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:09:17.370 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:09:17.371 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:17.371 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:19.001 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:20.003 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:20.003 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:20.003 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:09:20.004 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:09:20.004 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:20.004 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:21.676 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:22.678 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:22.678 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:22.678 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:09:22.679 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:09:22.679 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:22.679 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:23.420 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:24.422 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:24.423 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:24.423 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:09:24.423 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:09:24.423 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:24.424 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:24.921 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:25.922 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:25.922 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:25.922 INFO:tasks.workunit.client.0.smithi016.stdout:18 2024-09-15T21:09:25.923 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 18 2024-09-15T21:09:25.923 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:25.923 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:09:26.396 INFO:tasks.workunit.client.0.smithi016.stdout:osd.0 up in weight 1 up_from 4 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2213306305,v1:127.0.0.1:6801/2213306305] [v2:127.0.0.1:6802/2213306305,v1:127.0.0.1:6803/2213306305] exists,up 31048323-58bd-444d-882e-964d1933676a 2024-09-15T21:09:26.397 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:09:26.397 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:09:26.397 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:09:26.397 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:55: TEST_choose_args_update: ceph osd set-require-min-compat-client luminous 2024-09-15T21:09:27.100 INFO:tasks.workunit.client.0.smithi016.stderr:set require_min_compat_client to luminous 2024-09-15T21:09:27.124 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:56: TEST_choose_args_update: ceph osd getcrushmap 2024-09-15T21:09:27.583 INFO:tasks.workunit.client.0.smithi016.stderr:2 2024-09-15T21:09:27.601 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:57: TEST_choose_args_update: crushtool -d td/crush-choose-args/map -o td/crush-choose-args/map.txt 2024-09-15T21:09:27.625 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:58: TEST_choose_args_update: sed -i -e '/end crush map/d' td/crush-choose-args/map.txt 2024-09-15T21:09:27.626 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:59: TEST_choose_args_update: cat 2024-09-15T21:09:27.627 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:82: TEST_choose_args_update: crushtool -c td/crush-choose-args/map.txt -o td/crush-choose-args/map-new 2024-09-15T21:09:27.651 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:83: TEST_choose_args_update: ceph osd setcrushmap -i td/crush-choose-args/map-new 2024-09-15T21:09:28.296 INFO:tasks.workunit.client.0.smithi016.stderr:4 2024-09-15T21:09:28.319 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:84: TEST_choose_args_update: ceph osd crush tree 2024-09-15T21:09:28.759 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT 0 TYPE NAME 2024-09-15T21:09:28.759 INFO:tasks.workunit.client.0.smithi016.stdout:-1 3.00000 root default 2024-09-15T21:09:28.759 INFO:tasks.workunit.client.0.smithi016.stdout:-2 3.00000 2.00000 host HOST 2024-09-15T21:09:28.759 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 2.00000 osd.0 2024-09-15T21:09:28.777 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:86: TEST_choose_args_update: run_osd td/crush-choose-args 1 2024-09-15T21:09:28.777 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-choose-args 2024-09-15T21:09:28.777 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:09:28.777 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2024-09-15T21:09:28.777 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:09:28.778 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-choose-args/1 2024-09-15T21:09:28.778 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false ' 2024-09-15T21:09:28.778 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:09:28.778 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:09:28.778 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:09:28.778 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-choose-args/1' 2024-09-15T21:09:28.778 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-choose-args/1/journal' 2024-09-15T21:09:28.778 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:09:28.778 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:09:28.778 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-choose-args' 2024-09-15T21:09:28.778 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:09:28.778 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:09:28.778 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:09:28.779 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:09:28.779 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:09:28.779 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:09:28.779 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:09:28.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:09:28.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:09:28.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:09:28.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:09:28.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-choose-args/$name.log' 2024-09-15T21:09:28.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-choose-args/$name.pid' 2024-09-15T21:09:28.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:09:28.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:09:28.780 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:09:28.780 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:09:28.780 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:09:28.780 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:09:28.780 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-choose-args/1 2024-09-15T21:09:28.780 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:09:28.780 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ff2a342d-8691-4907-a960-e9f50f6732b4 2024-09-15T21:09:28.781 INFO:tasks.workunit.client.0.smithi016.stdout:add osd1 ff2a342d-8691-4907-a960-e9f50f6732b4 2024-09-15T21:09:28.781 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 ff2a342d-8691-4907-a960-e9f50f6732b4' 2024-09-15T21:09:28.781 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:09:28.804 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAITedmi7zPLxAAaQ26YZSiAg3+5SgaataxKg== 2024-09-15T21:09:28.804 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAITedmi7zPLxAAaQ26YZSiAg3+5SgaataxKg=="}' 2024-09-15T21:09:28.804 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ff2a342d-8691-4907-a960-e9f50f6732b4 -i td/crush-choose-args/1/new.json 2024-09-15T21:09:29.430 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:09:29.445 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-choose-args/1/new.json 2024-09-15T21:09:29.447 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/1 --osd-journal=td/crush-choose-args/1/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAITedmi7zPLxAAaQ26YZSiAg3+5SgaataxKg== --osd-uuid ff2a342d-8691-4907-a960-e9f50f6732b4 2024-09-15T21:09:29.477 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:09:29.471+0000 7f2c7a8097c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:09:29.479 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:09:29.475+0000 7f2c7a8097c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:09:29.480 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:09:29.475+0000 7f2c7a8097c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:09:29.481 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:09:29.475+0000 7f2c7a8097c0 -1 bdev(0x563a8b537c00 td/crush-choose-args/1/block) open stat got: (1) Operation not permitted 2024-09-15T21:09:29.481 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:09:29.475+0000 7f2c7a8097c0 -1 bluestore(td/crush-choose-args/1) _read_fsid unparsable uuid 2024-09-15T21:09:35.059 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-choose-args/1/keyring 2024-09-15T21:09:35.060 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:09:35.061 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd1 key to auth repository 2024-09-15T21:09:35.061 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2024-09-15T21:09:35.061 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-choose-args/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:09:35.676 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.1 2024-09-15T21:09:35.678 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2024-09-15T21:09:35.678 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/1 --osd-journal=td/crush-choose-args/1/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:09:35.678 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:09:35.678 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:09:35.678 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:09:35.713 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:09:35.707+0000 7f434b8ee7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:09:35.715 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:09:35.711+0000 7f434b8ee7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:09:35.717 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:09:35.711+0000 7f434b8ee7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:09:36.161 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2024-09-15T21:09:36.161 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:09:36.161 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2024-09-15T21:09:36.161 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:09:36.162 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:09:36.162 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:09:36.163 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:36.163 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:09:36.163 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:36.163 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:36.614 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:36.977 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:09:36.971+0000 7f434b8ee7c0 -1 Falling back to public interface 2024-09-15T21:09:37.615 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:09:37.615 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:37.616 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:37.616 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:09:37.616 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:37.616 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:38.130 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:39.131 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:39.133 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:39.133 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:09:39.134 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:09:39.134 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:39.134 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:39.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:39.789 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:09:39.783+0000 7f434b8ee7c0 -1 osd.1 0 log_to_monitors true 2024-09-15T21:09:40.592 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:40.592 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:40.592 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:09:40.593 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:09:40.593 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:40.593 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:41.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:42.092 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:42.092 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:42.093 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:09:42.093 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:09:42.093 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:42.093 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:42.568 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:43.570 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:43.571 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:43.571 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:09:43.572 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:09:43.572 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:43.572 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:44.022 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:45.023 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:45.024 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:45.024 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:09:45.024 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:09:45.025 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:45.025 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:45.509 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:46.512 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:46.513 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:46.513 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:09:46.513 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:09:46.513 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:46.513 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:46.970 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:47.971 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:47.972 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:47.972 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:09:47.973 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:09:47.973 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:47.973 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:48.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:49.459 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:49.459 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:49.459 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:09:49.460 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:09:49.460 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:49.460 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:49.927 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:50.928 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:50.929 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:50.929 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:09:50.929 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:09:50.929 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:50.930 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:51.419 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:52.420 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:52.420 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:52.420 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:09:52.421 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:09:52.421 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:52.421 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:53.588 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:54.590 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:54.590 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:54.590 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:09:54.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:09:54.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:54.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:55.401 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:56.402 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:56.403 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:56.403 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:09:56.403 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:09:56.403 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:56.404 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:57.015 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:58.017 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:58.017 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:58.017 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:09:58.018 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:09:58.018 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:58.018 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:09:58.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:09:59.673 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:09:59.674 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:09:59.674 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:09:59.674 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:09:59.674 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:09:59.675 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:10:00.277 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:01.279 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:01.279 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:01.279 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:10:01.279 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:10:01.280 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:01.280 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:10:01.772 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:02.774 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:02.774 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:02.775 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:10:02.775 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:10:02.775 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:02.775 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:10:03.268 INFO:tasks.workunit.client.0.smithi016.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2787319131,v1:127.0.0.1:6811/2787319131] [v2:127.0.0.1:6812/2787319131,v1:127.0.0.1:6813/2787319131] exists,up ff2a342d-8691-4907-a960-e9f50f6732b4 2024-09-15T21:10:03.269 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:10:03.269 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:10:03.269 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:10:03.269 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:87: TEST_choose_args_update: ceph osd crush tree 2024-09-15T21:10:03.759 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT 0 TYPE NAME 2024-09-15T21:10:03.759 INFO:tasks.workunit.client.0.smithi016.stdout:-1 6.00000 root default 2024-09-15T21:10:03.760 INFO:tasks.workunit.client.0.smithi016.stdout:-2 6.00000 5.00000 host HOST 2024-09-15T21:10:03.760 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 2.00000 osd.0 2024-09-15T21:10:03.760 INFO:tasks.workunit.client.0.smithi016.stdout: 1 3.00000 3.00000 osd.1 2024-09-15T21:10:03.777 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:88: TEST_choose_args_update: ceph osd getcrushmap 2024-09-15T21:10:04.302 INFO:tasks.workunit.client.0.smithi016.stderr:5 2024-09-15T21:10:04.319 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:89: TEST_choose_args_update: crushtool -d td/crush-choose-args/map-one-more -o td/crush-choose-args/map-one-more.txt 2024-09-15T21:10:04.342 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:90: TEST_choose_args_update: cat td/crush-choose-args/map-one-more.txt 2024-09-15T21:10:04.343 INFO:tasks.workunit.client.0.smithi016.stdout:# begin crush map 2024-09-15T21:10:04.343 INFO:tasks.workunit.client.0.smithi016.stdout:tunable choose_local_tries 0 2024-09-15T21:10:04.343 INFO:tasks.workunit.client.0.smithi016.stdout:tunable choose_local_fallback_tries 0 2024-09-15T21:10:04.343 INFO:tasks.workunit.client.0.smithi016.stdout:tunable choose_total_tries 50 2024-09-15T21:10:04.343 INFO:tasks.workunit.client.0.smithi016.stdout:tunable chooseleaf_descend_once 1 2024-09-15T21:10:04.343 INFO:tasks.workunit.client.0.smithi016.stdout:tunable chooseleaf_vary_r 1 2024-09-15T21:10:04.343 INFO:tasks.workunit.client.0.smithi016.stdout:tunable chooseleaf_stable 1 2024-09-15T21:10:04.343 INFO:tasks.workunit.client.0.smithi016.stdout:tunable straw_calc_version 1 2024-09-15T21:10:04.344 INFO:tasks.workunit.client.0.smithi016.stdout:tunable allowed_bucket_algs 54 2024-09-15T21:10:04.344 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-15T21:10:04.344 INFO:tasks.workunit.client.0.smithi016.stdout:# devices 2024-09-15T21:10:04.344 INFO:tasks.workunit.client.0.smithi016.stdout:device 0 osd.0 2024-09-15T21:10:04.344 INFO:tasks.workunit.client.0.smithi016.stdout:device 1 osd.1 2024-09-15T21:10:04.344 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-15T21:10:04.344 INFO:tasks.workunit.client.0.smithi016.stdout:# types 2024-09-15T21:10:04.344 INFO:tasks.workunit.client.0.smithi016.stdout:type 0 osd 2024-09-15T21:10:04.344 INFO:tasks.workunit.client.0.smithi016.stdout:type 1 host 2024-09-15T21:10:04.344 INFO:tasks.workunit.client.0.smithi016.stdout:type 2 chassis 2024-09-15T21:10:04.344 INFO:tasks.workunit.client.0.smithi016.stdout:type 3 rack 2024-09-15T21:10:04.345 INFO:tasks.workunit.client.0.smithi016.stdout:type 4 row 2024-09-15T21:10:04.345 INFO:tasks.workunit.client.0.smithi016.stdout:type 5 pdu 2024-09-15T21:10:04.345 INFO:tasks.workunit.client.0.smithi016.stdout:type 6 pod 2024-09-15T21:10:04.345 INFO:tasks.workunit.client.0.smithi016.stdout:type 7 room 2024-09-15T21:10:04.345 INFO:tasks.workunit.client.0.smithi016.stdout:type 8 datacenter 2024-09-15T21:10:04.345 INFO:tasks.workunit.client.0.smithi016.stdout:type 9 zone 2024-09-15T21:10:04.345 INFO:tasks.workunit.client.0.smithi016.stdout:type 10 region 2024-09-15T21:10:04.345 INFO:tasks.workunit.client.0.smithi016.stdout:type 11 root 2024-09-15T21:10:04.345 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-15T21:10:04.345 INFO:tasks.workunit.client.0.smithi016.stdout:# buckets 2024-09-15T21:10:04.345 INFO:tasks.workunit.client.0.smithi016.stdout:host HOST { 2024-09-15T21:10:04.345 INFO:tasks.workunit.client.0.smithi016.stdout: id -2 # do not change unnecessarily 2024-09-15T21:10:04.345 INFO:tasks.workunit.client.0.smithi016.stdout: # weight 6.00000 2024-09-15T21:10:04.345 INFO:tasks.workunit.client.0.smithi016.stdout: alg straw2 2024-09-15T21:10:04.346 INFO:tasks.workunit.client.0.smithi016.stdout: hash 0 # rjenkins1 2024-09-15T21:10:04.346 INFO:tasks.workunit.client.0.smithi016.stdout: item osd.0 weight 3.00000 2024-09-15T21:10:04.346 INFO:tasks.workunit.client.0.smithi016.stdout: item osd.1 weight 3.00000 2024-09-15T21:10:04.346 INFO:tasks.workunit.client.0.smithi016.stdout:} 2024-09-15T21:10:04.346 INFO:tasks.workunit.client.0.smithi016.stdout:root default { 2024-09-15T21:10:04.346 INFO:tasks.workunit.client.0.smithi016.stdout: id -1 # do not change unnecessarily 2024-09-15T21:10:04.346 INFO:tasks.workunit.client.0.smithi016.stdout: # weight 6.00000 2024-09-15T21:10:04.346 INFO:tasks.workunit.client.0.smithi016.stdout: alg straw2 2024-09-15T21:10:04.346 INFO:tasks.workunit.client.0.smithi016.stdout: hash 0 # rjenkins1 2024-09-15T21:10:04.346 INFO:tasks.workunit.client.0.smithi016.stdout: item HOST weight 6.00000 2024-09-15T21:10:04.346 INFO:tasks.workunit.client.0.smithi016.stdout:} 2024-09-15T21:10:04.346 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-15T21:10:04.347 INFO:tasks.workunit.client.0.smithi016.stdout:# rules 2024-09-15T21:10:04.347 INFO:tasks.workunit.client.0.smithi016.stdout:rule replicated_rule { 2024-09-15T21:10:04.347 INFO:tasks.workunit.client.0.smithi016.stdout: id 0 2024-09-15T21:10:04.347 INFO:tasks.workunit.client.0.smithi016.stdout: type replicated 2024-09-15T21:10:04.347 INFO:tasks.workunit.client.0.smithi016.stdout: step take default 2024-09-15T21:10:04.347 INFO:tasks.workunit.client.0.smithi016.stdout: step choose firstn 0 type osd 2024-09-15T21:10:04.347 INFO:tasks.workunit.client.0.smithi016.stdout: step emit 2024-09-15T21:10:04.347 INFO:tasks.workunit.client.0.smithi016.stdout:} 2024-09-15T21:10:04.347 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-15T21:10:04.347 INFO:tasks.workunit.client.0.smithi016.stdout:# choose_args 2024-09-15T21:10:04.347 INFO:tasks.workunit.client.0.smithi016.stdout:choose_args 0 { 2024-09-15T21:10:04.347 INFO:tasks.workunit.client.0.smithi016.stdout: { 2024-09-15T21:10:04.347 INFO:tasks.workunit.client.0.smithi016.stdout: bucket_id -1 2024-09-15T21:10:04.347 INFO:tasks.workunit.client.0.smithi016.stdout: weight_set [ 2024-09-15T21:10:04.348 INFO:tasks.workunit.client.0.smithi016.stdout: [ 5.00000 ] 2024-09-15T21:10:04.348 INFO:tasks.workunit.client.0.smithi016.stdout: [ 5.00000 ] 2024-09-15T21:10:04.348 INFO:tasks.workunit.client.0.smithi016.stdout: ] 2024-09-15T21:10:04.348 INFO:tasks.workunit.client.0.smithi016.stdout: ids [ -10 ] 2024-09-15T21:10:04.348 INFO:tasks.workunit.client.0.smithi016.stdout: } 2024-09-15T21:10:04.348 INFO:tasks.workunit.client.0.smithi016.stdout: { 2024-09-15T21:10:04.348 INFO:tasks.workunit.client.0.smithi016.stdout: bucket_id -2 2024-09-15T21:10:04.348 INFO:tasks.workunit.client.0.smithi016.stdout: weight_set [ 2024-09-15T21:10:04.348 INFO:tasks.workunit.client.0.smithi016.stdout: [ 2.00000 3.00000 ] 2024-09-15T21:10:04.348 INFO:tasks.workunit.client.0.smithi016.stdout: [ 2.00000 3.00000 ] 2024-09-15T21:10:04.348 INFO:tasks.workunit.client.0.smithi016.stdout: ] 2024-09-15T21:10:04.348 INFO:tasks.workunit.client.0.smithi016.stdout: ids [ -20 1 ] 2024-09-15T21:10:04.348 INFO:tasks.workunit.client.0.smithi016.stdout: } 2024-09-15T21:10:04.348 INFO:tasks.workunit.client.0.smithi016.stdout:} 2024-09-15T21:10:04.349 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-15T21:10:04.349 INFO:tasks.workunit.client.0.smithi016.stdout:# end crush map 2024-09-15T21:10:04.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:91: TEST_choose_args_update: diff -u td/crush-choose-args/map-one-more.txt /home/ubuntu/cephtest/clone.client.0/src/test/crush/crush-choose-args-expected-one-more-3.txt 2024-09-15T21:10:04.366 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:93: TEST_choose_args_update: destroy_osd td/crush-choose-args 1 2024-09-15T21:10:04.367 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:775: destroy_osd: local dir=td/crush-choose-args 2024-09-15T21:10:04.367 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:776: destroy_osd: local id=1 2024-09-15T21:10:04.367 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:778: destroy_osd: ceph osd out osd.1 2024-09-15T21:10:04.880 INFO:tasks.workunit.client.0.smithi016.stderr:osd.1 is already out. 2024-09-15T21:10:04.895 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:779: destroy_osd: kill_daemons td/crush-choose-args TERM osd.1 2024-09-15T21:10:04.896 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:10:04.896 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:10:04.897 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:10:04.897 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:10:04.897 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:10:05.206 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:10:05.206 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:780: destroy_osd: ceph osd down osd.1 2024-09-15T21:10:05.698 INFO:tasks.workunit.client.0.smithi016.stderr:osd.1 is already down. 2024-09-15T21:10:05.715 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:781: destroy_osd: ceph osd purge osd.1 --yes-i-really-mean-it 2024-09-15T21:10:06.206 INFO:tasks.workunit.client.0.smithi016.stderr:osd.1 does not exist 2024-09-15T21:10:06.223 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:782: destroy_osd: teardown td/crush-choose-args/1 2024-09-15T21:10:06.223 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-choose-args/1 2024-09-15T21:10:06.223 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:10:06.223 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-choose-args/1 KILL 2024-09-15T21:10:06.223 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:10:06.223 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:10:06.223 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:10:06.224 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:10:06.224 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:10:06.226 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:10:06.226 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:10:06.227 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:10:06.227 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:10:06.228 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:10:06.228 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:10:06.228 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:10:06.229 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:10:06.229 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:10:06.230 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:10:06.230 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:10:06.230 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:10:06.232 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:10:06.232 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-choose-args/1 2024-09-15T21:10:06.236 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:10:06.236 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:10:06.236 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:10:06.236 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.21400 2024-09-15T21:10:06.237 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:10:06.237 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:10:06.237 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:783: destroy_osd: rm -fr td/crush-choose-args/1 2024-09-15T21:10:06.238 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:94: TEST_choose_args_update: ceph osd crush tree 2024-09-15T21:10:06.701 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT 0 TYPE NAME 2024-09-15T21:10:06.701 INFO:tasks.workunit.client.0.smithi016.stdout:-1 3.00000 root default 2024-09-15T21:10:06.701 INFO:tasks.workunit.client.0.smithi016.stdout:-2 3.00000 2.00000 host HOST 2024-09-15T21:10:06.701 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 2.00000 osd.0 2024-09-15T21:10:06.718 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:95: TEST_choose_args_update: ceph osd getcrushmap 2024-09-15T21:10:07.185 INFO:tasks.workunit.client.0.smithi016.stderr:6 2024-09-15T21:10:07.202 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:96: TEST_choose_args_update: crushtool -d td/crush-choose-args/map-one-less -o td/crush-choose-args/map-one-less.txt 2024-09-15T21:10:07.224 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:97: TEST_choose_args_update: diff -u td/crush-choose-args/map-one-less.txt td/crush-choose-args/map.txt 2024-09-15T21:10:07.226 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:41: run: teardown td/crush-choose-args 2024-09-15T21:10:07.226 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-choose-args 2024-09-15T21:10:07.226 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:10:07.226 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-choose-args KILL 2024-09-15T21:10:07.227 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:10:07.227 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:10:07.227 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:10:07.227 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:10:07.227 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:10:07.335 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:10:07.335 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:10:07.336 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:10:07.336 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:10:07.337 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:10:07.337 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:10:07.338 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:10:07.338 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:10:07.338 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:10:07.339 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:10:07.339 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:10:07.339 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:10:07.340 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:10:07.340 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-choose-args 2024-09-15T21:10:07.347 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:10:07.348 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:10:07.348 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:10:07.348 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.21400 2024-09-15T21:10:07.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:10:07.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:10:07.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:38: run: for func in $funcs 2024-09-15T21:10:07.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:39: run: setup td/crush-choose-args 2024-09-15T21:10:07.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/crush-choose-args 2024-09-15T21:10:07.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/crush-choose-args 2024-09-15T21:10:07.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-choose-args 2024-09-15T21:10:07.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:10:07.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-choose-args KILL 2024-09-15T21:10:07.350 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:10:07.350 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:10:07.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:10:07.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:10:07.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:10:07.351 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:10:07.351 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:10:07.352 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:10:07.352 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:10:07.353 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:10:07.353 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:10:07.353 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:10:07.354 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:10:07.354 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:10:07.354 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:10:07.354 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:10:07.355 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:10:07.356 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:10:07.356 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-choose-args 2024-09-15T21:10:07.357 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:10:07.357 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:10:07.357 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:10:07.357 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.21400 2024-09-15T21:10:07.358 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:10:07.358 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:10:07.358 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/crush-choose-args 2024-09-15T21:10:07.359 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2024-09-15T21:10:07.359 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:10:07.359 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:10:07.359 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.21400 2024-09-15T21:10:07.361 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2024-09-15T21:10:07.361 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2024-09-15T21:10:07.361 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2024-09-15T21:10:07.361 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/crush-choose-args 1' TERM HUP INT 2024-09-15T21:10:07.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:40: run: TEST_move_bucket td/crush-choose-args 2024-09-15T21:10:07.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:195: TEST_move_bucket: local dir=td/crush-choose-args 2024-09-15T21:10:07.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:197: TEST_move_bucket: run_mon td/crush-choose-args a 2024-09-15T21:10:07.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/crush-choose-args 2024-09-15T21:10:07.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2024-09-15T21:10:07.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2024-09-15T21:10:07.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2024-09-15T21:10:07.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/crush-choose-args/a 2024-09-15T21:10:07.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/crush-choose-args/a --run-dir=td/crush-choose-args 2024-09-15T21:10:07.802 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2024-09-15T21:10:07.802 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:10:07.802 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:10:07.802 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:10:07.803 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:10:07.803 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:10:07.803 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:10:07.803 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/crush-choose-args/a '--log-file=td/crush-choose-args/$name.log' '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --mon-cluster-log-file=td/crush-choose-args/log --run-dir=td/crush-choose-args '--pid-file=td/crush-choose-args/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2024-09-15T21:10:08.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2024-09-15T21:10:08.040 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2024-09-15T21:10:08.040 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:10:08.040 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:10:08.041 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2024-09-15T21:10:08.041 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2024-09-15T21:10:08.041 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:10:08.041 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:10:08.041 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:10:08.042 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:10:08.042 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:10:08.042 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:10:08.042 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.21400/ceph-mon.a.asok 2024-09-15T21:10:08.043 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:10:08.043 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.21400/ceph-mon.a.asok config get fsid 2024-09-15T21:10:08.161 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2024-09-15T21:10:08.161 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:10:08.161 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:10:08.162 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2024-09-15T21:10:08.163 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2024-09-15T21:10:08.163 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:10:08.163 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:10:08.163 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:10:08.163 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:10:08.163 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:10:08.163 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:10:08.163 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.21400/ceph-mon.a.asok 2024-09-15T21:10:08.163 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:10:08.164 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.21400/ceph-mon.a.asok config get mon_host 2024-09-15T21:10:08.274 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:198: TEST_move_bucket: run_mgr td/crush-choose-args x 2024-09-15T21:10:08.274 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/crush-choose-args 2024-09-15T21:10:08.274 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2024-09-15T21:10:08.274 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2024-09-15T21:10:08.274 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2024-09-15T21:10:08.275 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/crush-choose-args/x 2024-09-15T21:10:08.275 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2024-09-15T21:10:09.199 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2024-09-15T21:10:09.199 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:10:09.199 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:10:09.199 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:10:09.199 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:10:09.199 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:10:09.199 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:10:09.200 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2024-09-15T21:10:09.200 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/crush-choose-args/x '--log-file=td/crush-choose-args/$name.log' '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --run-dir=td/crush-choose-args '--pid-file=td/crush-choose-args/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2024-09-15T21:10:09.263 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:199: TEST_move_bucket: run_osd td/crush-choose-args 0 2024-09-15T21:10:09.263 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-choose-args 2024-09-15T21:10:09.264 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:10:09.264 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2024-09-15T21:10:09.264 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:10:09.264 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-choose-args/0 2024-09-15T21:10:09.264 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false ' 2024-09-15T21:10:09.264 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:10:09.264 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:10:09.264 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:10:09.264 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-choose-args/0' 2024-09-15T21:10:09.264 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-choose-args/0/journal' 2024-09-15T21:10:09.265 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:10:09.265 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:10:09.265 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-choose-args' 2024-09-15T21:10:09.265 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:10:09.265 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:10:09.265 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:10:09.265 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:10:09.265 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:10:09.265 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:10:09.265 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:10:09.265 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:10:09.266 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:10:09.266 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:10:09.266 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:10:09.266 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-choose-args/$name.log' 2024-09-15T21:10:09.266 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-choose-args/$name.pid' 2024-09-15T21:10:09.266 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:10:09.266 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:10:09.266 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:10:09.266 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:10:09.266 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:10:09.267 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:10:09.267 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-choose-args/0 2024-09-15T21:10:09.267 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:10:09.267 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3c260d90-f12a-42c1-9202-e745b153c5cc 2024-09-15T21:10:09.267 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 3c260d90-f12a-42c1-9202-e745b153c5cc' 2024-09-15T21:10:09.267 INFO:tasks.workunit.client.0.smithi016.stdout:add osd0 3c260d90-f12a-42c1-9202-e745b153c5cc 2024-09-15T21:10:09.268 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:10:09.284 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAxTedmYPzaEBAAZJ8Bj1EjphafeEkRboi+Hg== 2024-09-15T21:10:09.284 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAxTedmYPzaEBAAZJ8Bj1EjphafeEkRboi+Hg=="}' 2024-09-15T21:10:09.285 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3c260d90-f12a-42c1-9202-e745b153c5cc -i td/crush-choose-args/0/new.json 2024-09-15T21:10:09.601 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:10:09.617 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-choose-args/0/new.json 2024-09-15T21:10:09.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/0 --osd-journal=td/crush-choose-args/0/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAxTedmYPzaEBAAZJ8Bj1EjphafeEkRboi+Hg== --osd-uuid 3c260d90-f12a-42c1-9202-e745b153c5cc 2024-09-15T21:10:09.649 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:09.642+0000 7f13f83a97c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:10:09.668 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:09.662+0000 7f13f83a97c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:10:09.670 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:09.662+0000 7f13f83a97c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:10:09.670 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:09.666+0000 7f13f83a97c0 -1 bdev(0x55c24f39fc00 td/crush-choose-args/0/block) open stat got: (1) Operation not permitted 2024-09-15T21:10:09.670 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:09.666+0000 7f13f83a97c0 -1 bluestore(td/crush-choose-args/0) _read_fsid unparsable uuid 2024-09-15T21:10:15.700 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-choose-args/0/keyring 2024-09-15T21:10:15.756 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:10:15.756 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2024-09-15T21:10:15.757 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd0 key to auth repository 2024-09-15T21:10:15.757 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-choose-args/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:10:16.310 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.0 2024-09-15T21:10:16.310 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2024-09-15T21:10:16.311 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/0 --osd-journal=td/crush-choose-args/0/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:10:16.311 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:10:16.311 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:10:16.311 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:10:16.345 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:16.338+0000 7f89443197c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:10:16.348 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:16.342+0000 7f89443197c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:10:16.349 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:16.342+0000 7f89443197c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:10:16.811 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2024-09-15T21:10:16.812 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:10:16.812 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2024-09-15T21:10:16.812 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:10:16.812 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:10:16.812 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:16.812 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:10:16.812 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:10:16.813 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:16.813 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:17.312 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:17.604 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:17.598+0000 7f89443197c0 -1 Falling back to public interface 2024-09-15T21:10:18.313 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:18.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:18.314 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:10:18.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:10:18.315 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:18.315 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:18.806 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:19.807 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:19.808 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:10:19.808 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:19.809 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:10:19.809 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:19.809 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:20.266 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:20.314 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:20.306+0000 7f89443197c0 -1 osd.0 0 log_to_monitors true 2024-09-15T21:10:21.268 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:21.269 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:21.269 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:10:21.269 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:10:21.270 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:21.270 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:21.717 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:22.718 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:22.718 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:10:22.719 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:22.719 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:10:22.719 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:22.719 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:23.170 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:24.171 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:24.172 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:24.172 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:10:24.172 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:10:24.173 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:24.173 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:24.615 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:25.617 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:25.617 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:25.618 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:10:25.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:10:25.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:25.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:26.071 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:27.073 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:27.073 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:27.073 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:10:27.074 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:10:27.074 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:27.074 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:27.536 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:28.538 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:28.538 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:28.538 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:10:28.539 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:10:28.539 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:28.539 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:29.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:30.053 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:30.053 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:30.054 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:10:30.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:10:30.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:30.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:30.534 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:31.536 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:31.536 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:31.537 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:10:31.537 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:10:31.537 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:31.537 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:32.166 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:33.168 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:33.168 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:33.169 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:10:33.169 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:10:33.169 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:33.169 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:33.863 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:34.865 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:34.865 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:34.865 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:10:34.866 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:10:34.866 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:34.866 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:35.414 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:36.416 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:36.416 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:36.416 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:10:36.417 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:10:36.417 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:36.417 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:37.377 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:38.379 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:38.379 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:38.379 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:10:38.380 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:10:38.380 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:38.380 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:39.229 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:40.230 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:40.230 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:40.231 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:10:40.231 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:10:40.231 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:40.231 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:40.933 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:41.934 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:41.935 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:41.935 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:10:41.935 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:10:41.935 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:41.936 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:42.616 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:43.617 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:43.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:43.618 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:10:43.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:10:43.619 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:43.619 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:10:44.104 INFO:tasks.workunit.client.0.smithi016.stdout:osd.0 up in weight 1 up_from 4 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2611952501,v1:127.0.0.1:6803/2611952501] [v2:127.0.0.1:6804/2611952501,v1:127.0.0.1:6805/2611952501] exists,up 3c260d90-f12a-42c1-9202-e745b153c5cc 2024-09-15T21:10:44.107 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:10:44.107 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:10:44.107 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:10:44.107 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:200: TEST_move_bucket: run_osd td/crush-choose-args 1 2024-09-15T21:10:44.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-choose-args 2024-09-15T21:10:44.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:10:44.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2024-09-15T21:10:44.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:10:44.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-choose-args/1 2024-09-15T21:10:44.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false ' 2024-09-15T21:10:44.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:10:44.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:10:44.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:10:44.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-choose-args/1' 2024-09-15T21:10:44.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-choose-args/1/journal' 2024-09-15T21:10:44.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:10:44.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:10:44.109 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-choose-args' 2024-09-15T21:10:44.109 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:10:44.109 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:10:44.109 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:10:44.109 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:10:44.109 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:10:44.109 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:10:44.109 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:10:44.109 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:10:44.109 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:10:44.109 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:10:44.109 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:10:44.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-choose-args/$name.log' 2024-09-15T21:10:44.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-choose-args/$name.pid' 2024-09-15T21:10:44.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:10:44.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:10:44.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:10:44.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:10:44.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:10:44.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:10:44.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-choose-args/1 2024-09-15T21:10:44.111 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:10:44.111 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1e81c143-86fd-4318-9c91-3a689b6bb1f0 2024-09-15T21:10:44.111 INFO:tasks.workunit.client.0.smithi016.stdout:add osd1 1e81c143-86fd-4318-9c91-3a689b6bb1f0 2024-09-15T21:10:44.111 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 1e81c143-86fd-4318-9c91-3a689b6bb1f0' 2024-09-15T21:10:44.111 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:10:44.131 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBUTedmm1y+BxAAsE4viNbRtI+lZB5lC/hfLQ== 2024-09-15T21:10:44.131 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBUTedmm1y+BxAAsE4viNbRtI+lZB5lC/hfLQ=="}' 2024-09-15T21:10:44.131 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1e81c143-86fd-4318-9c91-3a689b6bb1f0 -i td/crush-choose-args/1/new.json 2024-09-15T21:10:44.713 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:10:44.731 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-choose-args/1/new.json 2024-09-15T21:10:44.733 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/1 --osd-journal=td/crush-choose-args/1/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBUTedmm1y+BxAAsE4viNbRtI+lZB5lC/hfLQ== --osd-uuid 1e81c143-86fd-4318-9c91-3a689b6bb1f0 2024-09-15T21:10:44.772 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:44.766+0000 7f7ec38fc7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:10:44.785 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:44.778+0000 7f7ec38fc7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:10:44.787 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:44.782+0000 7f7ec38fc7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:10:44.788 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:44.782+0000 7f7ec38fc7c0 -1 bdev(0x56508aa07c00 td/crush-choose-args/1/block) open stat got: (1) Operation not permitted 2024-09-15T21:10:44.788 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:44.782+0000 7f7ec38fc7c0 -1 bluestore(td/crush-choose-args/1) _read_fsid unparsable uuid 2024-09-15T21:10:50.372 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-choose-args/1/keyring 2024-09-15T21:10:50.372 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:10:50.374 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd1 key to auth repository 2024-09-15T21:10:50.374 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2024-09-15T21:10:50.374 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-choose-args/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:10:50.975 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.1 2024-09-15T21:10:50.976 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2024-09-15T21:10:50.976 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/1 --osd-journal=td/crush-choose-args/1/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:10:50.976 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:10:50.976 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:10:50.977 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:10:51.003 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:50.998+0000 7fa9a22727c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:10:51.057 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:51.050+0000 7fa9a22727c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:10:51.058 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:51.054+0000 7fa9a22727c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:10:51.494 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2024-09-15T21:10:51.494 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:10:51.494 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2024-09-15T21:10:51.494 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:10:51.494 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:10:51.494 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:51.495 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:10:51.495 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:10:51.495 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:51.495 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:10:52.005 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:52.537 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:52.530+0000 7fa9a22727c0 -1 Falling back to public interface 2024-09-15T21:10:53.006 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:53.007 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:53.007 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:10:53.007 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:10:53.007 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:53.008 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:10:53.567 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:54.569 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:54.569 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:54.569 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:10:54.570 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:10:54.570 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:54.570 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:10:55.040 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:55.600 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:10:55.594+0000 7fa9a22727c0 -1 osd.1 0 log_to_monitors true 2024-09-15T21:10:56.041 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:56.042 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:56.042 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:10:56.042 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:10:56.042 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:56.043 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:10:56.554 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:57.556 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:57.556 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:57.556 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:10:57.557 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:10:57.557 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:57.557 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:10:58.012 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:10:59.016 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:10:59.016 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:10:59.017 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:10:59.017 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:10:59.017 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:10:59.017 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:10:59.590 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:00.592 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:00.592 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:00.592 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:11:00.593 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:11:00.593 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:00.593 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:11:01.234 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:02.235 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:02.235 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:02.236 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:11:02.236 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:11:02.236 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:02.236 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:11:02.790 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:03.792 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:03.793 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:03.793 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:11:03.793 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:11:03.793 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:03.794 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:11:04.288 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:05.290 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:05.290 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:05.290 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:11:05.291 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:11:05.291 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:05.291 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:11:05.753 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:06.755 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:06.755 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:06.756 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:11:06.756 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:11:06.756 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:06.756 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:11:07.327 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:08.329 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:08.330 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:08.330 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:11:08.330 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:11:08.330 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:08.331 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:11:09.045 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:10.047 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:10.047 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:10.047 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:11:10.048 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:11:10.048 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:10.048 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:11:10.525 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:11.528 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:11.529 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:11.529 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:11:11.529 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:11:11.529 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:11.530 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:11:12.197 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:13.199 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:13.199 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:13.200 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:11:13.200 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:11:13.200 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:13.200 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:11:14.254 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:15.256 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:15.256 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:15.257 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:11:15.257 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:11:15.257 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:15.257 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:11:15.995 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:16.997 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:16.997 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:16.997 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:11:16.998 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:11:16.998 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:16.998 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:11:17.942 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:18.944 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:18.945 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:18.945 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:11:18.945 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:11:18.945 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:18.945 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:11:19.485 INFO:tasks.workunit.client.0.smithi016.stdout:osd.1 up in weight 1 up_from 8 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2166286443,v1:127.0.0.1:6811/2166286443] [v2:127.0.0.1:6812/2166286443,v1:127.0.0.1:6813/2166286443] exists,up 1e81c143-86fd-4318-9c91-3a689b6bb1f0 2024-09-15T21:11:19.486 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:11:19.486 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:11:19.486 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:11:19.486 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:202: TEST_move_bucket: ceph osd crush weight-set create-compat 2024-09-15T21:11:20.169 INFO:tasks.workunit.client.0.smithi016.stderr:compat weight-set already created 2024-09-15T21:11:20.186 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:203: TEST_move_bucket: ceph osd crush weight-set reweight-compat osd.0 2 2024-09-15T21:11:20.935 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:204: TEST_move_bucket: ceph osd crush weight-set reweight-compat osd.1 2 2024-09-15T21:11:21.809 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:205: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:22.353 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT (compat) TYPE NAME 2024-09-15T21:11:22.353 INFO:tasks.workunit.client.0.smithi016.stdout:-1 6.00000 root default 2024-09-15T21:11:22.353 INFO:tasks.workunit.client.0.smithi016.stdout:-2 6.00000 4.00000 host HOST 2024-09-15T21:11:22.353 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 2.00000 osd.0 2024-09-15T21:11:22.353 INFO:tasks.workunit.client.0.smithi016.stdout: 1 3.00000 2.00000 osd.1 2024-09-15T21:11:22.373 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:206: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:22.373 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:206: TEST_move_bucket: grep '6.00000 4.00000' 2024-09-15T21:11:22.373 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:206: TEST_move_bucket: grep HOST 2024-09-15T21:11:22.867 INFO:tasks.workunit.client.0.smithi016.stdout:-2 6.00000 4.00000 host HOST 2024-09-15T21:11:22.867 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:209: TEST_move_bucket: ceph osd crush add-bucket RACK rack root=default 2024-09-15T21:11:23.431 INFO:tasks.workunit.client.0.smithi016.stderr:bucket 'RACK' already exists 2024-09-15T21:11:23.448 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:210: TEST_move_bucket: ceph osd crush move HOST rack=RACK 2024-09-15T21:11:23.980 INFO:tasks.workunit.client.0.smithi016.stderr:no need to move item id -2 name 'HOST' to location {rack=RACK} in crush map 2024-09-15T21:11:23.997 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:211: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:24.473 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT (compat) TYPE NAME 2024-09-15T21:11:24.473 INFO:tasks.workunit.client.0.smithi016.stdout:-1 6.00000 root default 2024-09-15T21:11:24.473 INFO:tasks.workunit.client.0.smithi016.stdout:-3 6.00000 4.00000 rack RACK 2024-09-15T21:11:24.473 INFO:tasks.workunit.client.0.smithi016.stdout:-2 6.00000 4.00000 host HOST 2024-09-15T21:11:24.473 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 2.00000 osd.0 2024-09-15T21:11:24.473 INFO:tasks.workunit.client.0.smithi016.stdout: 1 3.00000 2.00000 osd.1 2024-09-15T21:11:24.489 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:212: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:24.489 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:212: TEST_move_bucket: grep HOST 2024-09-15T21:11:24.489 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:212: TEST_move_bucket: grep '6.00000 4.00000' 2024-09-15T21:11:24.963 INFO:tasks.workunit.client.0.smithi016.stdout:-2 6.00000 4.00000 host HOST 2024-09-15T21:11:24.964 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:213: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:24.964 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:213: TEST_move_bucket: grep RACK 2024-09-15T21:11:24.964 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:213: TEST_move_bucket: grep '6.00000 4.00000' 2024-09-15T21:11:25.465 INFO:tasks.workunit.client.0.smithi016.stdout:-3 6.00000 4.00000 rack RACK 2024-09-15T21:11:25.466 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:216: TEST_move_bucket: ceph osd crush weight-set reweight-compat osd.0 1 2024-09-15T21:11:26.131 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:217: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:26.661 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT (compat) TYPE NAME 2024-09-15T21:11:26.661 INFO:tasks.workunit.client.0.smithi016.stdout:-1 6.00000 root default 2024-09-15T21:11:26.661 INFO:tasks.workunit.client.0.smithi016.stdout:-3 6.00000 3.00000 rack RACK 2024-09-15T21:11:26.661 INFO:tasks.workunit.client.0.smithi016.stdout:-2 6.00000 3.00000 host HOST 2024-09-15T21:11:26.661 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 1.00000 osd.0 2024-09-15T21:11:26.662 INFO:tasks.workunit.client.0.smithi016.stdout: 1 3.00000 2.00000 osd.1 2024-09-15T21:11:26.678 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:218: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:26.678 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:218: TEST_move_bucket: grep HOST 2024-09-15T21:11:26.678 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:218: TEST_move_bucket: grep '6.00000 3.00000' 2024-09-15T21:11:27.197 INFO:tasks.workunit.client.0.smithi016.stdout:-2 6.00000 3.00000 host HOST 2024-09-15T21:11:27.198 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:219: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:27.198 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:219: TEST_move_bucket: grep RACK 2024-09-15T21:11:27.198 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:219: TEST_move_bucket: grep '6.00000 3.00000' 2024-09-15T21:11:27.697 INFO:tasks.workunit.client.0.smithi016.stdout:-3 6.00000 3.00000 rack RACK 2024-09-15T21:11:27.697 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:222: TEST_move_bucket: ceph config set mon osd_crush_update_weight_set true 2024-09-15T21:11:28.219 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:223: TEST_move_bucket: ceph osd crush add-bucket FOO host root=default 2024-09-15T21:11:28.826 INFO:tasks.workunit.client.0.smithi016.stderr:bucket 'FOO' already exists 2024-09-15T21:11:28.845 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:224: TEST_move_bucket: ceph osd crush move osd.0 host=FOO 2024-09-15T21:11:29.399 INFO:tasks.workunit.client.0.smithi016.stderr:no need to move item id 0 name 'osd.0' to location {host=FOO} in crush map 2024-09-15T21:11:29.416 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:225: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:29.904 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT (compat) TYPE NAME 2024-09-15T21:11:29.905 INFO:tasks.workunit.client.0.smithi016.stdout:-1 6.00000 root default 2024-09-15T21:11:29.905 INFO:tasks.workunit.client.0.smithi016.stdout:-4 3.00000 3.00000 host FOO 2024-09-15T21:11:29.905 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 3.00000 osd.0 2024-09-15T21:11:29.905 INFO:tasks.workunit.client.0.smithi016.stdout:-3 3.00000 2.00000 rack RACK 2024-09-15T21:11:29.905 INFO:tasks.workunit.client.0.smithi016.stdout:-2 3.00000 2.00000 host HOST 2024-09-15T21:11:29.905 INFO:tasks.workunit.client.0.smithi016.stdout: 1 3.00000 2.00000 osd.1 2024-09-15T21:11:29.922 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:226: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:29.922 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:226: TEST_move_bucket: grep osd.0 2024-09-15T21:11:29.922 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:226: TEST_move_bucket: grep '3.00000 3.00000' 2024-09-15T21:11:30.449 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 3.00000 osd.0 2024-09-15T21:11:30.450 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:227: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:30.450 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:227: TEST_move_bucket: grep HOST 2024-09-15T21:11:30.451 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:227: TEST_move_bucket: grep '3.00000 2.00000' 2024-09-15T21:11:30.974 INFO:tasks.workunit.client.0.smithi016.stdout:-2 3.00000 2.00000 host HOST 2024-09-15T21:11:30.975 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:228: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:30.975 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:228: TEST_move_bucket: grep RACK 2024-09-15T21:11:30.975 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:228: TEST_move_bucket: grep '3.00000 2.00000' 2024-09-15T21:11:31.458 INFO:tasks.workunit.client.0.smithi016.stdout:-3 3.00000 2.00000 rack RACK 2024-09-15T21:11:31.458 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:231: TEST_move_bucket: ceph config set mon osd_crush_update_weight_set false 2024-09-15T21:11:31.972 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:232: TEST_move_bucket: ceph osd crush move osd.1 host=FOO 2024-09-15T21:11:32.599 INFO:tasks.workunit.client.0.smithi016.stderr:no need to move item id 1 name 'osd.1' to location {host=FOO} in crush map 2024-09-15T21:11:32.616 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:233: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:33.121 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT (compat) TYPE NAME 2024-09-15T21:11:33.121 INFO:tasks.workunit.client.0.smithi016.stdout:-1 6.00000 root default 2024-09-15T21:11:33.122 INFO:tasks.workunit.client.0.smithi016.stdout:-4 6.00000 3.00000 host FOO 2024-09-15T21:11:33.122 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 3.00000 osd.0 2024-09-15T21:11:33.122 INFO:tasks.workunit.client.0.smithi016.stdout: 1 3.00000 0 osd.1 2024-09-15T21:11:33.122 INFO:tasks.workunit.client.0.smithi016.stdout:-3 0 0 rack RACK 2024-09-15T21:11:33.122 INFO:tasks.workunit.client.0.smithi016.stdout:-2 0 0 host HOST 2024-09-15T21:11:33.140 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:234: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:33.140 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:234: TEST_move_bucket: grep osd.0 2024-09-15T21:11:33.140 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:234: TEST_move_bucket: grep '3.00000 3.00000' 2024-09-15T21:11:33.638 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 3.00000 osd.0 2024-09-15T21:11:33.639 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:235: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:33.639 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:235: TEST_move_bucket: grep '3.00000 0' 2024-09-15T21:11:33.639 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:235: TEST_move_bucket: grep osd.1 2024-09-15T21:11:34.141 INFO:tasks.workunit.client.0.smithi016.stdout: 1 3.00000 0 osd.1 2024-09-15T21:11:34.141 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:236: TEST_move_bucket: ceph osd crush tree 2024-09-15T21:11:34.141 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:236: TEST_move_bucket: grep FOO 2024-09-15T21:11:34.142 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:236: TEST_move_bucket: grep '6.00000 3.00000' 2024-09-15T21:11:34.710 INFO:tasks.workunit.client.0.smithi016.stdout:-4 6.00000 3.00000 host FOO 2024-09-15T21:11:34.712 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:41: run: teardown td/crush-choose-args 2024-09-15T21:11:34.712 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-choose-args 2024-09-15T21:11:34.712 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:11:34.712 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-choose-args KILL 2024-09-15T21:11:34.712 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:11:34.712 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:11:34.712 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:11:34.712 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:11:34.712 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:11:35.022 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:11:35.022 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:11:35.023 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:11:35.024 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:11:35.026 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:11:35.026 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:11:35.026 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:11:35.027 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:11:35.027 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:11:35.028 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:11:35.028 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:11:35.029 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:11:35.030 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:11:35.031 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-choose-args 2024-09-15T21:11:35.045 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:11:35.045 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:11:35.045 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:11:35.045 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.21400 2024-09-15T21:11:35.047 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:11:35.047 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:11:35.047 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:38: run: for func in $funcs 2024-09-15T21:11:35.047 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:39: run: setup td/crush-choose-args 2024-09-15T21:11:35.048 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/crush-choose-args 2024-09-15T21:11:35.048 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/crush-choose-args 2024-09-15T21:11:35.048 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-choose-args 2024-09-15T21:11:35.048 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:11:35.048 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-choose-args KILL 2024-09-15T21:11:35.048 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:11:35.048 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:11:35.048 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:11:35.048 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:11:35.048 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:11:35.050 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:11:35.050 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:11:35.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:11:35.051 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:11:35.053 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:11:35.053 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:11:35.053 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:11:35.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:11:35.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:11:35.055 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:11:35.055 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:11:35.056 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:11:35.058 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:11:35.058 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-choose-args 2024-09-15T21:11:35.059 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:11:35.059 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:11:35.060 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:11:35.060 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.21400 2024-09-15T21:11:35.061 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:11:35.061 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:11:35.062 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/crush-choose-args 2024-09-15T21:11:35.063 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2024-09-15T21:11:35.063 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:11:35.063 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:11:35.064 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.21400 2024-09-15T21:11:35.066 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2024-09-15T21:11:35.066 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2024-09-15T21:11:35.066 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2024-09-15T21:11:35.066 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/crush-choose-args 1' TERM HUP INT 2024-09-15T21:11:35.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:40: run: TEST_no_update_weight_set td/crush-choose-args 2024-09-15T21:11:35.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:104: TEST_no_update_weight_set: local dir=td/crush-choose-args 2024-09-15T21:11:35.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:106: TEST_no_update_weight_set: ORIG_CEPH_ARGS='--fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false ' 2024-09-15T21:11:35.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:107: TEST_no_update_weight_set: CEPH_ARGS+='--osd-crush-update-weight-set=false ' 2024-09-15T21:11:35.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:109: TEST_no_update_weight_set: run_mon td/crush-choose-args a 2024-09-15T21:11:35.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/crush-choose-args 2024-09-15T21:11:35.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2024-09-15T21:11:35.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2024-09-15T21:11:35.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2024-09-15T21:11:35.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/crush-choose-args/a 2024-09-15T21:11:35.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/crush-choose-args/a --run-dir=td/crush-choose-args 2024-09-15T21:11:35.567 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2024-09-15T21:11:35.567 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:11:35.568 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:11:35.568 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:11:35.568 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:11:35.568 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:11:35.568 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:11:35.569 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/crush-choose-args/a '--log-file=td/crush-choose-args/$name.log' '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --mon-cluster-log-file=td/crush-choose-args/log --run-dir=td/crush-choose-args '--pid-file=td/crush-choose-args/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2024-09-15T21:11:35.778 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2024-09-15T21:11:35.779 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2024-09-15T21:11:35.779 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:11:35.779 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:11:35.780 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2024-09-15T21:11:35.780 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2024-09-15T21:11:35.780 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:11:35.780 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:11:35.780 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:11:35.781 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:11:35.781 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:11:35.781 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:11:35.781 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.21400/ceph-mon.a.asok 2024-09-15T21:11:35.782 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:11:35.782 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.21400/ceph-mon.a.asok config get fsid 2024-09-15T21:11:35.902 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2024-09-15T21:11:35.903 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:11:35.903 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:11:35.903 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2024-09-15T21:11:35.903 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2024-09-15T21:11:35.903 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:11:35.904 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:11:35.904 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:11:35.904 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:11:35.904 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:11:35.904 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:11:35.905 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.21400/ceph-mon.a.asok 2024-09-15T21:11:35.905 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:11:35.905 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.21400/ceph-mon.a.asok config get mon_host 2024-09-15T21:11:36.007 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:110: TEST_no_update_weight_set: run_mgr td/crush-choose-args x 2024-09-15T21:11:36.007 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/crush-choose-args 2024-09-15T21:11:36.008 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2024-09-15T21:11:36.008 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2024-09-15T21:11:36.008 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2024-09-15T21:11:36.008 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/crush-choose-args/x 2024-09-15T21:11:36.008 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2024-09-15T21:11:36.921 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2024-09-15T21:11:36.921 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:11:36.921 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:11:36.922 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:11:36.922 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:11:36.922 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:11:36.922 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:11:36.922 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2024-09-15T21:11:36.923 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/crush-choose-args/x '--log-file=td/crush-choose-args/$name.log' '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --run-dir=td/crush-choose-args '--pid-file=td/crush-choose-args/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2024-09-15T21:11:36.985 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:111: TEST_no_update_weight_set: run_osd td/crush-choose-args 0 2024-09-15T21:11:36.985 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-choose-args 2024-09-15T21:11:36.985 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:11:36.985 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2024-09-15T21:11:36.985 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:11:36.985 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-choose-args/0 2024-09-15T21:11:36.985 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false ' 2024-09-15T21:11:36.985 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:11:36.985 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:11:36.985 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:11:36.985 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-choose-args/0' 2024-09-15T21:11:36.986 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-choose-args/0/journal' 2024-09-15T21:11:36.986 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:11:36.986 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:11:36.986 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-choose-args' 2024-09-15T21:11:36.986 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:11:36.986 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:11:36.986 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:11:36.986 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:11:36.986 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:11:36.986 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:11:36.986 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:11:36.988 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:11:36.988 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:11:36.988 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:11:36.988 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:11:36.988 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-choose-args/$name.log' 2024-09-15T21:11:36.988 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-choose-args/$name.pid' 2024-09-15T21:11:36.988 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:11:36.988 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:11:36.988 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:11:36.988 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:11:36.988 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:11:36.989 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:11:36.989 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-choose-args/0 2024-09-15T21:11:36.989 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:11:36.989 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8eadeab3-c27a-4da9-93ac-0ad480e0bda8 2024-09-15T21:11:36.989 INFO:tasks.workunit.client.0.smithi016.stdout:add osd0 8eadeab3-c27a-4da9-93ac-0ad480e0bda8 2024-09-15T21:11:36.989 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8eadeab3-c27a-4da9-93ac-0ad480e0bda8' 2024-09-15T21:11:36.989 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:11:37.010 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCJTedmtqOKABAAMzUVmtCkwg56jMwb9E34/g== 2024-09-15T21:11:37.010 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCJTedmtqOKABAAMzUVmtCkwg56jMwb9E34/g=="}' 2024-09-15T21:11:37.010 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8eadeab3-c27a-4da9-93ac-0ad480e0bda8 -i td/crush-choose-args/0/new.json 2024-09-15T21:11:37.307 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:11:37.322 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-choose-args/0/new.json 2024-09-15T21:11:37.323 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/0 --osd-journal=td/crush-choose-args/0/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCJTedmtqOKABAAMzUVmtCkwg56jMwb9E34/g== --osd-uuid 8eadeab3-c27a-4da9-93ac-0ad480e0bda8 2024-09-15T21:11:37.364 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:11:37.358+0000 7f0e9c67c7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:11:37.375 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:11:37.370+0000 7f0e9c67c7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:11:37.376 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:11:37.370+0000 7f0e9c67c7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:11:37.377 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:11:37.374+0000 7f0e9c67c7c0 -1 bdev(0x5560c304dc00 td/crush-choose-args/0/block) open stat got: (1) Operation not permitted 2024-09-15T21:11:37.377 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:11:37.374+0000 7f0e9c67c7c0 -1 bluestore(td/crush-choose-args/0) _read_fsid unparsable uuid 2024-09-15T21:11:43.234 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-choose-args/0/keyring 2024-09-15T21:11:43.235 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:11:43.237 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd0 key to auth repository 2024-09-15T21:11:43.237 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2024-09-15T21:11:43.237 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-choose-args/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:11:43.827 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.0 2024-09-15T21:11:43.827 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2024-09-15T21:11:43.827 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:11:43.828 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:11:43.828 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:11:43.828 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/0 --osd-journal=td/crush-choose-args/0/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:11:43.866 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:11:43.862+0000 7f88f2f2a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:11:43.868 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:11:43.862+0000 7f88f2f2a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:11:43.869 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:11:43.866+0000 7f88f2f2a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:11:44.354 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2024-09-15T21:11:44.354 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:11:44.354 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2024-09-15T21:11:44.354 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:11:44.354 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:11:44.354 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:44.354 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:11:44.355 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:11:44.355 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:44.355 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:11:44.845 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:44.891 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:11:44.886+0000 7f88f2f2a7c0 -1 Falling back to public interface 2024-09-15T21:11:45.846 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:45.846 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:45.847 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:11:45.847 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:11:45.847 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:45.847 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:11:46.331 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:47.332 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:47.332 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:47.332 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:11:47.333 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:11:47.333 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:47.333 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:11:47.468 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:11:47.462+0000 7f88f2f2a7c0 -1 osd.0 0 log_to_monitors true 2024-09-15T21:11:47.937 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:48.939 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:48.939 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:48.939 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:11:48.939 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:11:48.940 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:48.940 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:11:49.405 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:50.407 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:50.407 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:50.407 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:11:50.408 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:11:50.408 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:50.408 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:11:50.863 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:51.865 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:51.865 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:51.865 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:11:51.865 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:11:51.866 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:51.866 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:11:52.439 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:53.441 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:53.441 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:53.441 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:11:53.442 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:11:53.442 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:53.442 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:11:53.905 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:54.907 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:54.907 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:54.907 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:11:54.908 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:11:54.908 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:54.908 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:11:55.489 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:56.491 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:56.491 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:56.492 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:11:56.492 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:11:56.492 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:56.492 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:11:57.066 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:58.068 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:58.068 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:58.068 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:11:58.068 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:11:58.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:58.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:11:58.515 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:11:59.517 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:11:59.517 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:11:59.517 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:11:59.518 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:11:59.518 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:11:59.518 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:12:00.115 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:01.116 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:01.117 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:01.117 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:12:01.117 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:12:01.117 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:01.117 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:12:02.115 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:03.117 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:03.117 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:03.118 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:12:03.118 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:12:03.118 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:03.118 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:12:03.880 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:04.882 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:04.882 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:04.882 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:12:04.882 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:12:04.883 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:04.883 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:12:05.638 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:06.640 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:06.640 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:06.640 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:12:06.641 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:12:06.641 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:06.641 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:12:07.179 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:08.181 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:08.181 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:08.182 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:12:08.182 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:12:08.182 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:08.182 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:12:09.320 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:10.322 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:10.322 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:10.323 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:12:10.323 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:12:10.324 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:10.324 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:12:10.786 INFO:tasks.workunit.client.0.smithi016.stdout:osd.0 up in weight 1 up_from 4 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/319767926,v1:127.0.0.1:6803/319767926] [v2:127.0.0.1:6804/319767926,v1:127.0.0.1:6805/319767926] exists,up 8eadeab3-c27a-4da9-93ac-0ad480e0bda8 2024-09-15T21:12:10.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:12:10.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:12:10.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:12:10.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:113: TEST_no_update_weight_set: ceph osd set-require-min-compat-client luminous 2024-09-15T21:12:11.434 INFO:tasks.workunit.client.0.smithi016.stderr:set require_min_compat_client to luminous 2024-09-15T21:12:11.453 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:114: TEST_no_update_weight_set: ceph osd crush tree 2024-09-15T21:12:11.939 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT TYPE NAME 2024-09-15T21:12:11.939 INFO:tasks.workunit.client.0.smithi016.stdout:-1 3.00000 root default 2024-09-15T21:12:11.939 INFO:tasks.workunit.client.0.smithi016.stdout:-2 3.00000 host HOST 2024-09-15T21:12:11.939 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 osd.0 2024-09-15T21:12:11.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:115: TEST_no_update_weight_set: ceph osd getcrushmap 2024-09-15T21:12:12.465 INFO:tasks.workunit.client.0.smithi016.stderr:2 2024-09-15T21:12:12.485 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:116: TEST_no_update_weight_set: crushtool -d td/crush-choose-args/map -o td/crush-choose-args/map.txt 2024-09-15T21:12:12.510 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:117: TEST_no_update_weight_set: sed -i -e '/end crush map/d' td/crush-choose-args/map.txt 2024-09-15T21:12:12.513 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:118: TEST_no_update_weight_set: cat 2024-09-15T21:12:12.514 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:141: TEST_no_update_weight_set: crushtool -c td/crush-choose-args/map.txt -o td/crush-choose-args/map-new 2024-09-15T21:12:12.538 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:142: TEST_no_update_weight_set: ceph osd setcrushmap -i td/crush-choose-args/map-new 2024-09-15T21:12:13.206 INFO:tasks.workunit.client.0.smithi016.stderr:4 2024-09-15T21:12:13.229 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:143: TEST_no_update_weight_set: ceph osd crush tree 2024-09-15T21:12:13.745 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT 0 TYPE NAME 2024-09-15T21:12:13.746 INFO:tasks.workunit.client.0.smithi016.stdout:-1 3.00000 root default 2024-09-15T21:12:13.746 INFO:tasks.workunit.client.0.smithi016.stdout:-2 3.00000 2.00000 host HOST 2024-09-15T21:12:13.746 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 2.00000 osd.0 2024-09-15T21:12:13.763 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:146: TEST_no_update_weight_set: run_osd td/crush-choose-args 1 2024-09-15T21:12:13.763 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-choose-args 2024-09-15T21:12:13.763 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:12:13.763 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2024-09-15T21:12:13.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:12:13.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-choose-args/1 2024-09-15T21:12:13.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false ' 2024-09-15T21:12:13.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:12:13.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:12:13.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:12:13.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-choose-args/1' 2024-09-15T21:12:13.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-choose-args/1/journal' 2024-09-15T21:12:13.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:12:13.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:12:13.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-choose-args' 2024-09-15T21:12:13.764 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:12:13.764 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:12:13.765 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:12:13.765 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:12:13.765 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:12:13.765 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:12:13.767 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:12:13.767 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:12:13.767 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:12:13.767 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:12:13.767 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:12:13.767 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-choose-args/$name.log' 2024-09-15T21:12:13.767 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-choose-args/$name.pid' 2024-09-15T21:12:13.767 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:12:13.767 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:12:13.767 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:12:13.768 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:12:13.768 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:12:13.768 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:12:13.768 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-choose-args/1 2024-09-15T21:12:13.768 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:12:13.768 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=57585025-61ee-484b-9e13-757ae4f741ae 2024-09-15T21:12:13.768 INFO:tasks.workunit.client.0.smithi016.stdout:add osd1 57585025-61ee-484b-9e13-757ae4f741ae 2024-09-15T21:12:13.769 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 57585025-61ee-484b-9e13-757ae4f741ae' 2024-09-15T21:12:13.769 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:12:13.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCtTedmLQdsLxAALGpyN7AbR/BrISLoO8+04A== 2024-09-15T21:12:13.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCtTedmLQdsLxAALGpyN7AbR/BrISLoO8+04A=="}' 2024-09-15T21:12:13.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 57585025-61ee-484b-9e13-757ae4f741ae -i td/crush-choose-args/1/new.json 2024-09-15T21:12:14.306 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:12:14.322 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-choose-args/1/new.json 2024-09-15T21:12:14.323 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/1 --osd-journal=td/crush-choose-args/1/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCtTedmLQdsLxAALGpyN7AbR/BrISLoO8+04A== --osd-uuid 57585025-61ee-484b-9e13-757ae4f741ae 2024-09-15T21:12:14.363 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:14.361+0000 7f7f454807c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:12:14.365 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:14.361+0000 7f7f454807c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:12:14.366 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:14.365+0000 7f7f454807c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:12:14.366 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:14.365+0000 7f7f454807c0 -1 bdev(0x55c6157f7c00 td/crush-choose-args/1/block) open stat got: (1) Operation not permitted 2024-09-15T21:12:14.367 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:14.365+0000 7f7f454807c0 -1 bluestore(td/crush-choose-args/1) _read_fsid unparsable uuid 2024-09-15T21:12:19.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-choose-args/1/keyring 2024-09-15T21:12:19.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:12:19.674 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd1 key to auth repository 2024-09-15T21:12:19.674 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2024-09-15T21:12:19.674 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-choose-args/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:12:20.298 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.1 2024-09-15T21:12:20.298 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2024-09-15T21:12:20.298 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/1 --osd-journal=td/crush-choose-args/1/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:12:20.299 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:12:20.299 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:12:20.299 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:12:20.337 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:20.333+0000 7f5e526c77c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:12:20.339 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:20.337+0000 7f5e526c77c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:12:20.340 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:20.337+0000 7f5e526c77c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:12:20.770 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2024-09-15T21:12:20.770 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:12:20.770 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2024-09-15T21:12:20.770 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:12:20.770 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:12:20.770 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:20.771 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:12:20.771 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:12:20.771 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:20.771 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:21.247 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:21.388 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:21.385+0000 7f5e526c77c0 -1 Falling back to public interface 2024-09-15T21:12:22.248 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:22.248 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:22.248 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:12:22.249 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:12:22.249 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:22.249 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:22.732 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:23.733 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:23.780 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:23.780 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:12:23.780 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:12:23.781 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:23.781 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:24.214 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:24.213+0000 7f5e526c77c0 -1 osd.1 0 log_to_monitors true 2024-09-15T21:12:24.269 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:25.271 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:25.271 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:25.271 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:12:25.272 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:12:25.272 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:25.272 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:25.912 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:26.914 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:26.914 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:26.914 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:12:26.914 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:12:26.915 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:26.915 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:27.377 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:28.379 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:28.379 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:28.379 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:12:28.380 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:12:28.380 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:28.380 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:28.938 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:29.939 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:29.940 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:29.940 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:12:29.941 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:12:29.941 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:29.941 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:30.529 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:31.530 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:31.530 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:31.530 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:12:31.531 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:12:31.531 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:31.531 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:31.985 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:32.987 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:32.987 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:32.987 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:12:32.988 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:12:32.988 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:32.988 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:33.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:34.435 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:34.435 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:34.435 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:12:34.436 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:12:34.436 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:34.436 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:34.955 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:35.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:35.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:35.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:12:35.958 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:12:35.958 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:35.958 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:36.532 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:37.533 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:37.534 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:37.534 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:12:37.534 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:12:37.534 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:37.534 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:38.524 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:39.526 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:39.526 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:39.526 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:12:39.527 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:12:39.527 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:39.527 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:40.387 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:41.389 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:41.389 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:41.390 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:12:41.390 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:12:41.390 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:41.390 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:42.185 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:43.186 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:43.186 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:43.186 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:12:43.187 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:12:43.187 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:43.187 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:44.009 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:45.011 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:45.011 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:45.012 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:12:45.012 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:12:45.012 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:45.012 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:45.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:46.620 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:46.620 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:46.620 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:12:46.620 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:12:46.621 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:46.621 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:47.112 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:12:48.114 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:12:48.114 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:12:48.114 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:12:48.114 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:12:48.115 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:12:48.115 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:12:48.651 INFO:tasks.workunit.client.0.smithi016.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1976008628,v1:127.0.0.1:6811/1976008628] [v2:127.0.0.1:6812/1976008628,v1:127.0.0.1:6813/1976008628] exists,up 57585025-61ee-484b-9e13-757ae4f741ae 2024-09-15T21:12:48.651 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:12:48.651 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:12:48.652 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:12:48.652 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:147: TEST_no_update_weight_set: ceph osd crush tree 2024-09-15T21:12:49.142 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT 0 TYPE NAME 2024-09-15T21:12:49.142 INFO:tasks.workunit.client.0.smithi016.stdout:-1 6.00000 root default 2024-09-15T21:12:49.142 INFO:tasks.workunit.client.0.smithi016.stdout:-2 6.00000 2.00000 host HOST 2024-09-15T21:12:49.142 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 2.00000 osd.0 2024-09-15T21:12:49.142 INFO:tasks.workunit.client.0.smithi016.stdout: 1 3.00000 0 osd.1 2024-09-15T21:12:49.160 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:148: TEST_no_update_weight_set: ceph osd getcrushmap 2024-09-15T21:12:49.728 INFO:tasks.workunit.client.0.smithi016.stderr:5 2024-09-15T21:12:49.745 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:149: TEST_no_update_weight_set: crushtool -d td/crush-choose-args/map-one-more -o td/crush-choose-args/map-one-more.txt 2024-09-15T21:12:49.772 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:150: TEST_no_update_weight_set: cat td/crush-choose-args/map-one-more.txt 2024-09-15T21:12:49.772 INFO:tasks.workunit.client.0.smithi016.stdout:# begin crush map 2024-09-15T21:12:49.773 INFO:tasks.workunit.client.0.smithi016.stdout:tunable choose_local_tries 0 2024-09-15T21:12:49.773 INFO:tasks.workunit.client.0.smithi016.stdout:tunable choose_local_fallback_tries 0 2024-09-15T21:12:49.773 INFO:tasks.workunit.client.0.smithi016.stdout:tunable choose_total_tries 50 2024-09-15T21:12:49.773 INFO:tasks.workunit.client.0.smithi016.stdout:tunable chooseleaf_descend_once 1 2024-09-15T21:12:49.773 INFO:tasks.workunit.client.0.smithi016.stdout:tunable chooseleaf_vary_r 1 2024-09-15T21:12:49.773 INFO:tasks.workunit.client.0.smithi016.stdout:tunable chooseleaf_stable 1 2024-09-15T21:12:49.773 INFO:tasks.workunit.client.0.smithi016.stdout:tunable straw_calc_version 1 2024-09-15T21:12:49.773 INFO:tasks.workunit.client.0.smithi016.stdout:tunable allowed_bucket_algs 54 2024-09-15T21:12:49.773 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-15T21:12:49.773 INFO:tasks.workunit.client.0.smithi016.stdout:# devices 2024-09-15T21:12:49.773 INFO:tasks.workunit.client.0.smithi016.stdout:device 0 osd.0 2024-09-15T21:12:49.773 INFO:tasks.workunit.client.0.smithi016.stdout:device 1 osd.1 2024-09-15T21:12:49.773 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-15T21:12:49.774 INFO:tasks.workunit.client.0.smithi016.stdout:# types 2024-09-15T21:12:49.774 INFO:tasks.workunit.client.0.smithi016.stdout:type 0 osd 2024-09-15T21:12:49.774 INFO:tasks.workunit.client.0.smithi016.stdout:type 1 host 2024-09-15T21:12:49.774 INFO:tasks.workunit.client.0.smithi016.stdout:type 2 chassis 2024-09-15T21:12:49.774 INFO:tasks.workunit.client.0.smithi016.stdout:type 3 rack 2024-09-15T21:12:49.774 INFO:tasks.workunit.client.0.smithi016.stdout:type 4 row 2024-09-15T21:12:49.774 INFO:tasks.workunit.client.0.smithi016.stdout:type 5 pdu 2024-09-15T21:12:49.774 INFO:tasks.workunit.client.0.smithi016.stdout:type 6 pod 2024-09-15T21:12:49.774 INFO:tasks.workunit.client.0.smithi016.stdout:type 7 room 2024-09-15T21:12:49.774 INFO:tasks.workunit.client.0.smithi016.stdout:type 8 datacenter 2024-09-15T21:12:49.774 INFO:tasks.workunit.client.0.smithi016.stdout:type 9 zone 2024-09-15T21:12:49.774 INFO:tasks.workunit.client.0.smithi016.stdout:type 10 region 2024-09-15T21:12:49.774 INFO:tasks.workunit.client.0.smithi016.stdout:type 11 root 2024-09-15T21:12:49.775 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-15T21:12:49.775 INFO:tasks.workunit.client.0.smithi016.stdout:# buckets 2024-09-15T21:12:49.775 INFO:tasks.workunit.client.0.smithi016.stdout:host HOST { 2024-09-15T21:12:49.775 INFO:tasks.workunit.client.0.smithi016.stdout: id -2 # do not change unnecessarily 2024-09-15T21:12:49.775 INFO:tasks.workunit.client.0.smithi016.stdout: # weight 6.00000 2024-09-15T21:12:49.775 INFO:tasks.workunit.client.0.smithi016.stdout: alg straw2 2024-09-15T21:12:49.775 INFO:tasks.workunit.client.0.smithi016.stdout: hash 0 # rjenkins1 2024-09-15T21:12:49.775 INFO:tasks.workunit.client.0.smithi016.stdout: item osd.0 weight 3.00000 2024-09-15T21:12:49.775 INFO:tasks.workunit.client.0.smithi016.stdout: item osd.1 weight 3.00000 2024-09-15T21:12:49.776 INFO:tasks.workunit.client.0.smithi016.stdout:} 2024-09-15T21:12:49.776 INFO:tasks.workunit.client.0.smithi016.stdout:root default { 2024-09-15T21:12:49.776 INFO:tasks.workunit.client.0.smithi016.stdout: id -1 # do not change unnecessarily 2024-09-15T21:12:49.776 INFO:tasks.workunit.client.0.smithi016.stdout: # weight 6.00000 2024-09-15T21:12:49.776 INFO:tasks.workunit.client.0.smithi016.stdout: alg straw2 2024-09-15T21:12:49.776 INFO:tasks.workunit.client.0.smithi016.stdout: hash 0 # rjenkins1 2024-09-15T21:12:49.776 INFO:tasks.workunit.client.0.smithi016.stdout: item HOST weight 6.00000 2024-09-15T21:12:49.776 INFO:tasks.workunit.client.0.smithi016.stdout:} 2024-09-15T21:12:49.776 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-15T21:12:49.776 INFO:tasks.workunit.client.0.smithi016.stdout:# rules 2024-09-15T21:12:49.776 INFO:tasks.workunit.client.0.smithi016.stdout:rule replicated_rule { 2024-09-15T21:12:49.776 INFO:tasks.workunit.client.0.smithi016.stdout: id 0 2024-09-15T21:12:49.776 INFO:tasks.workunit.client.0.smithi016.stdout: type replicated 2024-09-15T21:12:49.777 INFO:tasks.workunit.client.0.smithi016.stdout: step take default 2024-09-15T21:12:49.777 INFO:tasks.workunit.client.0.smithi016.stdout: step choose firstn 0 type osd 2024-09-15T21:12:49.777 INFO:tasks.workunit.client.0.smithi016.stdout: step emit 2024-09-15T21:12:49.777 INFO:tasks.workunit.client.0.smithi016.stdout:} 2024-09-15T21:12:49.777 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-15T21:12:49.777 INFO:tasks.workunit.client.0.smithi016.stdout:# choose_args 2024-09-15T21:12:49.777 INFO:tasks.workunit.client.0.smithi016.stdout:choose_args 0 { 2024-09-15T21:12:49.777 INFO:tasks.workunit.client.0.smithi016.stdout: { 2024-09-15T21:12:49.777 INFO:tasks.workunit.client.0.smithi016.stdout: bucket_id -1 2024-09-15T21:12:49.777 INFO:tasks.workunit.client.0.smithi016.stdout: weight_set [ 2024-09-15T21:12:49.777 INFO:tasks.workunit.client.0.smithi016.stdout: [ 2.00000 ] 2024-09-15T21:12:49.777 INFO:tasks.workunit.client.0.smithi016.stdout: [ 1.00000 ] 2024-09-15T21:12:49.777 INFO:tasks.workunit.client.0.smithi016.stdout: ] 2024-09-15T21:12:49.777 INFO:tasks.workunit.client.0.smithi016.stdout: ids [ -10 ] 2024-09-15T21:12:49.778 INFO:tasks.workunit.client.0.smithi016.stdout: } 2024-09-15T21:12:49.778 INFO:tasks.workunit.client.0.smithi016.stdout: { 2024-09-15T21:12:49.778 INFO:tasks.workunit.client.0.smithi016.stdout: bucket_id -2 2024-09-15T21:12:49.778 INFO:tasks.workunit.client.0.smithi016.stdout: weight_set [ 2024-09-15T21:12:49.778 INFO:tasks.workunit.client.0.smithi016.stdout: [ 2.00000 0.00000 ] 2024-09-15T21:12:49.778 INFO:tasks.workunit.client.0.smithi016.stdout: [ 1.00000 0.00000 ] 2024-09-15T21:12:49.778 INFO:tasks.workunit.client.0.smithi016.stdout: ] 2024-09-15T21:12:49.778 INFO:tasks.workunit.client.0.smithi016.stdout: ids [ -20 1 ] 2024-09-15T21:12:49.778 INFO:tasks.workunit.client.0.smithi016.stdout: } 2024-09-15T21:12:49.778 INFO:tasks.workunit.client.0.smithi016.stdout:} 2024-09-15T21:12:49.778 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-15T21:12:49.778 INFO:tasks.workunit.client.0.smithi016.stdout:# end crush map 2024-09-15T21:12:49.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:151: TEST_no_update_weight_set: diff -u td/crush-choose-args/map-one-more.txt /home/ubuntu/cephtest/clone.client.0/src/test/crush/crush-choose-args-expected-one-more-0.txt 2024-09-15T21:12:49.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:153: TEST_no_update_weight_set: destroy_osd td/crush-choose-args 1 2024-09-15T21:12:49.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:775: destroy_osd: local dir=td/crush-choose-args 2024-09-15T21:12:49.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:776: destroy_osd: local id=1 2024-09-15T21:12:49.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:778: destroy_osd: ceph osd out osd.1 2024-09-15T21:12:50.381 INFO:tasks.workunit.client.0.smithi016.stderr:osd.1 is already out. 2024-09-15T21:12:50.399 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:779: destroy_osd: kill_daemons td/crush-choose-args TERM osd.1 2024-09-15T21:12:50.400 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:12:50.400 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:12:50.401 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:12:50.401 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:12:50.401 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:12:50.711 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:12:50.711 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:780: destroy_osd: ceph osd down osd.1 2024-09-15T21:12:51.191 INFO:tasks.workunit.client.0.smithi016.stderr:osd.1 is already down. 2024-09-15T21:12:51.208 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:781: destroy_osd: ceph osd purge osd.1 --yes-i-really-mean-it 2024-09-15T21:12:51.709 INFO:tasks.workunit.client.0.smithi016.stderr:osd.1 does not exist 2024-09-15T21:12:51.727 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:782: destroy_osd: teardown td/crush-choose-args/1 2024-09-15T21:12:51.728 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-choose-args/1 2024-09-15T21:12:51.728 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:12:51.728 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-choose-args/1 KILL 2024-09-15T21:12:51.728 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:12:51.729 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:12:51.729 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:12:51.729 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:12:51.729 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:12:51.731 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:12:51.732 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:12:51.733 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:12:51.733 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:12:51.734 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:12:51.734 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:12:51.735 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:12:51.736 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:12:51.736 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:12:51.736 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:12:51.737 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:12:51.738 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:12:51.739 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:12:51.739 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-choose-args/1 2024-09-15T21:12:51.743 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:12:51.744 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:12:51.744 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:12:51.744 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.21400 2024-09-15T21:12:51.744 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:12:51.745 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:12:51.745 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:783: destroy_osd: rm -fr td/crush-choose-args/1 2024-09-15T21:12:51.745 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:154: TEST_no_update_weight_set: ceph osd crush tree 2024-09-15T21:12:52.295 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT 0 TYPE NAME 2024-09-15T21:12:52.295 INFO:tasks.workunit.client.0.smithi016.stdout:-1 3.00000 root default 2024-09-15T21:12:52.296 INFO:tasks.workunit.client.0.smithi016.stdout:-2 3.00000 2.00000 host HOST 2024-09-15T21:12:52.296 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 2.00000 osd.0 2024-09-15T21:12:52.312 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:155: TEST_no_update_weight_set: ceph osd getcrushmap 2024-09-15T21:12:52.788 INFO:tasks.workunit.client.0.smithi016.stderr:6 2024-09-15T21:12:52.806 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:156: TEST_no_update_weight_set: crushtool -d td/crush-choose-args/map-one-less -o td/crush-choose-args/map-one-less.txt 2024-09-15T21:12:52.833 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:157: TEST_no_update_weight_set: diff -u td/crush-choose-args/map-one-less.txt td/crush-choose-args/map.txt 2024-09-15T21:12:52.834 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:159: TEST_no_update_weight_set: CEPH_ARGS='--fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false ' 2024-09-15T21:12:52.835 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:41: run: teardown td/crush-choose-args 2024-09-15T21:12:52.835 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-choose-args 2024-09-15T21:12:52.835 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:12:52.835 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-choose-args KILL 2024-09-15T21:12:52.835 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:12:52.836 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:12:52.836 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:12:52.836 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:12:52.836 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:12:53.146 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:12:53.147 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:12:53.147 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:12:53.148 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:12:53.150 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:12:53.150 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:12:53.150 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:12:53.152 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:12:53.152 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:12:53.153 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:12:53.153 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:12:53.154 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:12:53.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:12:53.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-choose-args 2024-09-15T21:12:53.165 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:12:53.165 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:12:53.165 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:12:53.166 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.21400 2024-09-15T21:12:53.167 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:12:53.167 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:12:53.167 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:38: run: for func in $funcs 2024-09-15T21:12:53.168 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:39: run: setup td/crush-choose-args 2024-09-15T21:12:53.168 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/crush-choose-args 2024-09-15T21:12:53.168 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/crush-choose-args 2024-09-15T21:12:53.168 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-choose-args 2024-09-15T21:12:53.168 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:12:53.168 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-choose-args KILL 2024-09-15T21:12:53.168 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:12:53.168 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:12:53.169 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:12:53.169 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:12:53.169 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:12:53.170 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:12:53.171 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:12:53.173 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:12:53.173 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:12:53.174 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:12:53.175 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:12:53.175 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:12:53.176 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:12:53.176 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:12:53.176 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:12:53.177 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:12:53.177 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:12:53.179 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:12:53.179 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-choose-args 2024-09-15T21:12:53.180 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:12:53.180 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:12:53.181 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:12:53.181 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.21400 2024-09-15T21:12:53.182 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:12:53.182 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:12:53.182 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/crush-choose-args 2024-09-15T21:12:53.183 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2024-09-15T21:12:53.184 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:12:53.184 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:12:53.184 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.21400 2024-09-15T21:12:53.187 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2024-09-15T21:12:53.187 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2024-09-15T21:12:53.187 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2024-09-15T21:12:53.187 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/crush-choose-args 1' TERM HUP INT 2024-09-15T21:12:53.187 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:40: run: TEST_reweight td/crush-choose-args 2024-09-15T21:12:53.187 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:164: TEST_reweight: local dir=td/crush-choose-args 2024-09-15T21:12:53.187 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:166: TEST_reweight: ORIG_CEPH_ARGS='--fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false ' 2024-09-15T21:12:53.187 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:167: TEST_reweight: CEPH_ARGS+='--osd-crush-update-weight-set=false ' 2024-09-15T21:12:53.187 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:169: TEST_reweight: run_mon td/crush-choose-args a 2024-09-15T21:12:53.187 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/crush-choose-args 2024-09-15T21:12:53.188 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2024-09-15T21:12:53.188 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2024-09-15T21:12:53.188 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2024-09-15T21:12:53.188 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/crush-choose-args/a 2024-09-15T21:12:53.188 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/crush-choose-args/a --run-dir=td/crush-choose-args 2024-09-15T21:12:53.671 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2024-09-15T21:12:53.671 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:12:53.671 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:12:53.672 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:12:53.672 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:12:53.672 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:12:53.672 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:12:53.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/crush-choose-args/a '--log-file=td/crush-choose-args/$name.log' '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --mon-cluster-log-file=td/crush-choose-args/log --run-dir=td/crush-choose-args '--pid-file=td/crush-choose-args/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2024-09-15T21:12:53.900 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2024-09-15T21:12:53.900 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2024-09-15T21:12:53.901 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:12:53.901 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:12:53.901 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2024-09-15T21:12:53.902 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2024-09-15T21:12:53.902 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:12:53.902 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:12:53.902 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:12:53.902 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:12:53.902 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:12:53.902 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:12:53.902 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.21400/ceph-mon.a.asok 2024-09-15T21:12:53.903 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:12:53.903 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.21400/ceph-mon.a.asok config get fsid 2024-09-15T21:12:54.018 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2024-09-15T21:12:54.018 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:12:54.019 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:12:54.019 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2024-09-15T21:12:54.019 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2024-09-15T21:12:54.019 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:12:54.019 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:12:54.019 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:12:54.020 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:12:54.020 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:12:54.020 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:12:54.020 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.21400/ceph-mon.a.asok 2024-09-15T21:12:54.021 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:12:54.021 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.21400/ceph-mon.a.asok config get mon_host 2024-09-15T21:12:54.131 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:170: TEST_reweight: run_mgr td/crush-choose-args x 2024-09-15T21:12:54.132 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/crush-choose-args 2024-09-15T21:12:54.132 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2024-09-15T21:12:54.132 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2024-09-15T21:12:54.132 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2024-09-15T21:12:54.132 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/crush-choose-args/x 2024-09-15T21:12:54.132 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2024-09-15T21:12:55.049 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2024-09-15T21:12:55.049 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:12:55.049 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:12:55.050 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:12:55.050 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:12:55.050 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:12:55.051 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:12:55.051 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2024-09-15T21:12:55.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/crush-choose-args/x '--log-file=td/crush-choose-args/$name.log' '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --run-dir=td/crush-choose-args '--pid-file=td/crush-choose-args/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2024-09-15T21:12:55.089 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:171: TEST_reweight: run_osd td/crush-choose-args 0 2024-09-15T21:12:55.089 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-choose-args 2024-09-15T21:12:55.089 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:12:55.089 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2024-09-15T21:12:55.089 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:12:55.089 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-choose-args/0 2024-09-15T21:12:55.089 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false ' 2024-09-15T21:12:55.089 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:12:55.089 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:12:55.089 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:12:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-choose-args/0' 2024-09-15T21:12:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-choose-args/0/journal' 2024-09-15T21:12:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:12:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:12:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-choose-args' 2024-09-15T21:12:55.090 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:12:55.090 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:12:55.090 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:12:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:12:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:12:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:12:55.090 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:12:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:12:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:12:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:12:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:12:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-choose-args/$name.log' 2024-09-15T21:12:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-choose-args/$name.pid' 2024-09-15T21:12:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:12:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:12:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:12:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:12:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:12:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:12:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-choose-args/0 2024-09-15T21:12:55.092 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:12:55.092 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cbd9214b-a1ac-47f9-8198-e5fb759613c6 2024-09-15T21:12:55.092 INFO:tasks.workunit.client.0.smithi016.stdout:add osd0 cbd9214b-a1ac-47f9-8198-e5fb759613c6 2024-09-15T21:12:55.093 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 cbd9214b-a1ac-47f9-8198-e5fb759613c6' 2024-09-15T21:12:55.093 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:12:55.113 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDXTedmWUa7BhAAGL2ymZAh09yu0/cW/l1EEQ== 2024-09-15T21:12:55.113 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDXTedmWUa7BhAAGL2ymZAh09yu0/cW/l1EEQ=="}' 2024-09-15T21:12:55.113 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cbd9214b-a1ac-47f9-8198-e5fb759613c6 -i td/crush-choose-args/0/new.json 2024-09-15T21:12:55.402 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:12:55.417 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-choose-args/0/new.json 2024-09-15T21:12:55.418 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/0 --osd-journal=td/crush-choose-args/0/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDXTedmWUa7BhAAGL2ymZAh09yu0/cW/l1EEQ== --osd-uuid cbd9214b-a1ac-47f9-8198-e5fb759613c6 2024-09-15T21:12:55.457 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:55.457+0000 7f323e08a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:12:55.471 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:55.469+0000 7f323e08a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:12:55.472 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:55.469+0000 7f323e08a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:12:55.472 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:55.469+0000 7f323e08a7c0 -1 bdev(0x55d2924f3c00 td/crush-choose-args/0/block) open stat got: (1) Operation not permitted 2024-09-15T21:12:55.473 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:12:55.469+0000 7f323e08a7c0 -1 bluestore(td/crush-choose-args/0) _read_fsid unparsable uuid 2024-09-15T21:13:01.605 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-choose-args/0/keyring 2024-09-15T21:13:01.606 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:13:01.607 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd0 key to auth repository 2024-09-15T21:13:01.608 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2024-09-15T21:13:01.608 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-choose-args/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:13:02.197 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.0 2024-09-15T21:13:02.197 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2024-09-15T21:13:02.197 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/0 --osd-journal=td/crush-choose-args/0/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:13:02.198 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:13:02.198 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:13:02.198 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:13:02.234 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:02.233+0000 7f5ad58797c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:13:02.236 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:02.233+0000 7f5ad58797c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:13:02.237 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:02.237+0000 7f5ad58797c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:13:02.675 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2024-09-15T21:13:02.675 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:13:02.675 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2024-09-15T21:13:02.675 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:13:02.675 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:13:02.675 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:02.675 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:13:02.676 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:13:02.676 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:02.676 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:03.156 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:03.288 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:03.285+0000 7f5ad58797c0 -1 Falling back to public interface 2024-09-15T21:13:04.158 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:04.159 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:04.159 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:13:04.159 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:13:04.159 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:04.160 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:04.651 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:05.653 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:05.653 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:05.653 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:13:05.654 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:13:05.654 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:05.654 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:05.952 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:05.949+0000 7f5ad58797c0 -1 osd.0 0 log_to_monitors true 2024-09-15T21:13:06.220 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:07.222 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:07.222 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:07.222 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:13:07.223 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:13:07.223 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:07.223 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:07.700 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:08.701 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:08.702 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:08.702 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:13:08.702 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:13:08.703 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:08.703 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:09.182 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:10.184 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:10.184 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:10.185 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:13:10.185 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:13:10.185 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:10.185 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:10.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:11.674 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:11.674 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:11.674 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:13:11.675 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:13:11.675 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:11.675 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:12.140 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:13.142 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:13.142 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:13.142 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:13:13.143 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:13:13.143 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:13.143 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:13.694 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:14.696 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:14.696 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:14.697 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:13:14.697 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:13:14.697 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:14.697 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:15.165 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:16.166 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:16.166 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:16.167 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:13:16.167 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:13:16.167 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:16.167 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:16.758 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:17.760 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:17.760 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:17.760 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:13:17.760 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:13:17.761 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:17.761 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:18.441 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:19.442 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:19.442 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:19.443 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:13:19.443 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:13:19.443 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:19.443 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:20.399 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:21.401 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:21.401 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:21.401 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:13:21.401 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:13:21.402 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:21.402 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:22.189 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:23.191 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:23.191 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:23.191 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:13:23.192 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:13:23.192 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:23.192 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:23.826 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:24.829 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:24.829 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:24.829 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:13:24.829 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:13:24.830 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:24.830 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:25.559 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:26.561 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:26.561 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:26.561 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:13:26.562 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:13:26.562 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:26.562 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:27.453 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:28.455 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:28.455 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:28.456 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:13:28.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:13:28.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:28.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:13:28.950 INFO:tasks.workunit.client.0.smithi016.stdout:osd.0 up in weight 1 up_from 4 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2569332772,v1:127.0.0.1:6803/2569332772] [v2:127.0.0.1:6804/2569332772,v1:127.0.0.1:6805/2569332772] exists,up cbd9214b-a1ac-47f9-8198-e5fb759613c6 2024-09-15T21:13:28.954 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:13:28.954 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:13:28.954 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:13:28.954 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:172: TEST_reweight: run_osd td/crush-choose-args 1 2024-09-15T21:13:28.954 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-choose-args 2024-09-15T21:13:28.954 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:13:28.954 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2024-09-15T21:13:28.954 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:13:28.954 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-choose-args/1 2024-09-15T21:13:28.955 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false ' 2024-09-15T21:13:28.955 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:13:28.955 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:13:28.955 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:13:28.955 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-choose-args/1' 2024-09-15T21:13:28.955 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-choose-args/1/journal' 2024-09-15T21:13:28.955 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:13:28.955 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:13:28.955 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-choose-args' 2024-09-15T21:13:28.955 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:13:28.955 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:13:28.955 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:13:28.956 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:13:28.956 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:13:28.956 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:13:28.956 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:13:28.956 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:13:28.956 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:13:28.956 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:13:28.956 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:13:28.956 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-choose-args/$name.log' 2024-09-15T21:13:28.956 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-choose-args/$name.pid' 2024-09-15T21:13:28.956 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:13:28.956 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:13:28.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:13:28.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:13:28.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:13:28.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:13:28.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-choose-args/1 2024-09-15T21:13:28.957 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:13:28.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5e9e94b4-f8df-4125-b91e-d375f1ca5f9b 2024-09-15T21:13:28.958 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 5e9e94b4-f8df-4125-b91e-d375f1ca5f9b' 2024-09-15T21:13:28.958 INFO:tasks.workunit.client.0.smithi016.stdout:add osd1 5e9e94b4-f8df-4125-b91e-d375f1ca5f9b 2024-09-15T21:13:28.958 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:13:28.982 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD4Tedm9AGMOhAAZgHemFq+rZm4j4ctx8F6Sw== 2024-09-15T21:13:28.982 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD4Tedm9AGMOhAAZgHemFq+rZm4j4ctx8F6Sw=="}' 2024-09-15T21:13:28.982 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5e9e94b4-f8df-4125-b91e-d375f1ca5f9b -i td/crush-choose-args/1/new.json 2024-09-15T21:13:29.501 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:13:29.518 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-choose-args/1/new.json 2024-09-15T21:13:29.519 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/1 --osd-journal=td/crush-choose-args/1/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD4Tedm9AGMOhAAZgHemFq+rZm4j4ctx8F6Sw== --osd-uuid 5e9e94b4-f8df-4125-b91e-d375f1ca5f9b 2024-09-15T21:13:29.559 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:29.557+0000 7fd12e2027c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:13:29.582 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:29.581+0000 7fd12e2027c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:13:29.584 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:29.581+0000 7fd12e2027c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:13:29.584 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:29.581+0000 7fd12e2027c0 -1 bdev(0x5607ed575c00 td/crush-choose-args/1/block) open stat got: (1) Operation not permitted 2024-09-15T21:13:29.584 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:29.581+0000 7fd12e2027c0 -1 bluestore(td/crush-choose-args/1) _read_fsid unparsable uuid 2024-09-15T21:13:35.508 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-choose-args/1/keyring 2024-09-15T21:13:35.508 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:13:35.509 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2024-09-15T21:13:35.510 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd1 key to auth repository 2024-09-15T21:13:35.510 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-choose-args/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:13:36.110 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.1 2024-09-15T21:13:36.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2024-09-15T21:13:36.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:13:36.111 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/1 --osd-journal=td/crush-choose-args/1/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:13:36.111 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:13:36.111 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:13:36.149 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:36.145+0000 7f7c2fe977c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:13:36.153 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:36.153+0000 7f7c2fe977c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:13:36.154 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:36.153+0000 7f7c2fe977c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:13:36.590 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2024-09-15T21:13:36.590 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:13:36.590 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2024-09-15T21:13:36.590 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:13:36.590 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:13:36.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:36.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:13:36.591 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:13:36.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:36.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:37.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:37.167 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:37.165+0000 7f7c2fe977c0 -1 Falling back to public interface 2024-09-15T21:13:38.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:38.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:38.069 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:13:38.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:13:38.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:38.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:38.529 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:39.530 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:39.530 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:39.530 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:13:39.531 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:13:39.531 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:39.531 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:39.791 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:13:39.789+0000 7f7c2fe977c0 -1 osd.1 0 log_to_monitors true 2024-09-15T21:13:40.101 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:41.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:41.103 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:41.103 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:13:41.103 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:13:41.104 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:41.104 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:41.560 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:42.562 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:42.562 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:42.562 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:13:42.563 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:13:42.563 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:42.563 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:43.021 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:44.023 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:44.023 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:44.023 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:13:44.024 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:13:44.024 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:44.024 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:44.623 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:45.625 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:45.625 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:45.625 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:13:45.626 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:13:45.626 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:45.626 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:46.101 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:47.103 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:47.103 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:47.104 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:13:47.104 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:13:47.104 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:47.104 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:47.557 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:48.559 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:48.559 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:48.560 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:13:48.560 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:13:48.560 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:48.561 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:49.015 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:50.017 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:50.017 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:50.017 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:13:50.018 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:13:50.018 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:50.018 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:50.659 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:51.661 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:51.661 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:51.661 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:13:51.662 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:13:51.662 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:51.662 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:52.295 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:53.297 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:53.298 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:53.298 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:13:53.298 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:13:53.298 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:53.299 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:54.037 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:55.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:55.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:55.039 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:13:55.040 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:13:55.040 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:55.040 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:55.999 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:57.001 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:57.001 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:57.002 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:13:57.002 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:13:57.002 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:57.002 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:58.107 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:13:59.109 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:13:59.109 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:13:59.109 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:13:59.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:13:59.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:13:59.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:13:59.908 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:00.910 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:00.910 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:00.910 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:14:00.911 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:14:00.911 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:00.911 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:14:01.627 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:02.629 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:02.629 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:02.629 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:14:02.629 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:14:02.630 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:02.630 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:14:03.111 INFO:tasks.workunit.client.0.smithi016.stdout:osd.1 up in weight 1 up_from 8 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2684784968,v1:127.0.0.1:6811/2684784968] [v2:127.0.0.1:6812/2684784968,v1:127.0.0.1:6813/2684784968] exists,up 5e9e94b4-f8df-4125-b91e-d375f1ca5f9b 2024-09-15T21:14:03.112 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:14:03.112 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:14:03.112 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:14:03.112 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:174: TEST_reweight: ceph osd crush weight-set create-compat 2024-09-15T21:14:03.663 INFO:tasks.workunit.client.0.smithi016.stderr:compat weight-set already created 2024-09-15T21:14:03.679 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:175: TEST_reweight: ceph osd crush tree 2024-09-15T21:14:04.153 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT (compat) TYPE NAME 2024-09-15T21:14:04.153 INFO:tasks.workunit.client.0.smithi016.stdout:-1 6.00000 root default 2024-09-15T21:14:04.153 INFO:tasks.workunit.client.0.smithi016.stdout:-2 6.00000 6.00000 host HOST 2024-09-15T21:14:04.153 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 3.00000 osd.0 2024-09-15T21:14:04.153 INFO:tasks.workunit.client.0.smithi016.stdout: 1 3.00000 3.00000 osd.1 2024-09-15T21:14:04.169 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:177: TEST_reweight: ceph osd crush weight-set reweight-compat osd.0 2 2024-09-15T21:14:04.840 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:178: TEST_reweight: ceph osd crush tree 2024-09-15T21:14:05.327 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT (compat) TYPE NAME 2024-09-15T21:14:05.327 INFO:tasks.workunit.client.0.smithi016.stdout:-1 6.00000 root default 2024-09-15T21:14:05.327 INFO:tasks.workunit.client.0.smithi016.stdout:-2 6.00000 5.00000 host HOST 2024-09-15T21:14:05.327 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 2.00000 osd.0 2024-09-15T21:14:05.327 INFO:tasks.workunit.client.0.smithi016.stdout: 1 3.00000 3.00000 osd.1 2024-09-15T21:14:05.344 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:179: TEST_reweight: ceph osd crush tree 2024-09-15T21:14:05.344 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:179: TEST_reweight: grep host 2024-09-15T21:14:05.344 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:179: TEST_reweight: grep '6.00000 5.00000' 2024-09-15T21:14:05.864 INFO:tasks.workunit.client.0.smithi016.stdout:-2 6.00000 5.00000 host HOST 2024-09-15T21:14:05.867 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:181: TEST_reweight: run_osd td/crush-choose-args 2 2024-09-15T21:14:05.867 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-choose-args 2024-09-15T21:14:05.867 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:14:05.867 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2024-09-15T21:14:05.867 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:14:05.867 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-choose-args/2 2024-09-15T21:14:05.867 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false ' 2024-09-15T21:14:05.867 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:14:05.867 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:14:05.867 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:14:05.867 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-choose-args/2' 2024-09-15T21:14:05.868 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-choose-args/2/journal' 2024-09-15T21:14:05.868 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:14:05.868 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:14:05.868 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-choose-args' 2024-09-15T21:14:05.868 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:14:05.868 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:14:05.868 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:14:05.868 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:14:05.868 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:14:05.868 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:14:05.868 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:14:05.868 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' 2024-09-15T21:14:05.869 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:14:05.869 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:14:05.869 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:14:05.869 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-choose-args/$name.log' 2024-09-15T21:14:05.869 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-choose-args/$name.pid' 2024-09-15T21:14:05.869 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:14:05.869 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:14:05.869 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:14:05.869 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:14:05.869 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:14:05.869 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:14:05.869 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-choose-args/2 2024-09-15T21:14:05.870 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:14:05.870 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=eed50a73-30aa-4b6e-9bf9-4af5e2710c0b 2024-09-15T21:14:05.870 INFO:tasks.workunit.client.0.smithi016.stdout:add osd2 eed50a73-30aa-4b6e-9bf9-4af5e2710c0b 2024-09-15T21:14:05.871 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 eed50a73-30aa-4b6e-9bf9-4af5e2710c0b' 2024-09-15T21:14:05.871 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:14:05.890 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAdTudmbXcVNRAAcT+QO6eu6uDMdkjX9ELayw== 2024-09-15T21:14:05.891 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAdTudmbXcVNRAAcT+QO6eu6uDMdkjX9ELayw=="}' 2024-09-15T21:14:05.891 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new eed50a73-30aa-4b6e-9bf9-4af5e2710c0b -i td/crush-choose-args/2/new.json 2024-09-15T21:14:06.464 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:14:06.483 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-choose-args/2/new.json 2024-09-15T21:14:06.484 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/2 --osd-journal=td/crush-choose-args/2/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAdTudmbXcVNRAAcT+QO6eu6uDMdkjX9ELayw== --osd-uuid eed50a73-30aa-4b6e-9bf9-4af5e2710c0b 2024-09-15T21:14:06.525 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:06.524+0000 7fab0dd347c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:14:06.545 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:06.544+0000 7fab0dd347c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:14:06.547 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:06.544+0000 7fab0dd347c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:14:06.547 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:06.544+0000 7fab0dd347c0 -1 bdev(0x55d07c0edc00 td/crush-choose-args/2/block) open stat got: (1) Operation not permitted 2024-09-15T21:14:06.547 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:06.544+0000 7fab0dd347c0 -1 bluestore(td/crush-choose-args/2) _read_fsid unparsable uuid 2024-09-15T21:14:12.163 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-choose-args/2/keyring 2024-09-15T21:14:12.163 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:14:12.165 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2024-09-15T21:14:12.165 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd2 key to auth repository 2024-09-15T21:14:12.165 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-choose-args/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:14:12.764 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.2 2024-09-15T21:14:12.765 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2024-09-15T21:14:12.765 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:14:12.765 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:14:12.765 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8a5b76eb-0855-43b2-b926-3e75d4a536cf --auth-supported=none --mon-host=127.0.0.1:7131 --crush-location=root=default,host=HOST --osd-crush-initial-weight=3 --osd-class-update-on-start=false --osd-crush-update-weight-set=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-choose-args/2 --osd-journal=td/crush-choose-args/2/journal --chdir= --run-dir=td/crush-choose-args '--admin-socket=/tmp/ceph-asok.21400/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-choose-args/$name.log' '--pid-file=td/crush-choose-args/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:14:12.765 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:14:12.802 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:12.800+0000 7f9ce0c2c7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:14:12.817 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:12.816+0000 7f9ce0c2c7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:14:12.818 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:12.816+0000 7f9ce0c2c7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:14:13.246 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2024-09-15T21:14:13.246 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:14:13.246 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2024-09-15T21:14:13.246 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:14:13.247 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:14:13.247 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:14:13.247 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:13.247 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:14:13.247 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:13.247 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:13.721 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:14.091 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:14.088+0000 7f9ce0c2c7c0 -1 Falling back to public interface 2024-09-15T21:14:14.722 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:14.723 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:14.723 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:14:14.723 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:14:14.723 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:14.723 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:15.207 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:16.209 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:16.209 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:16.209 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:14:16.210 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:14:16.210 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:16.210 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:16.679 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:16.921 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:16.896+0000 7f9ce0c2c7c0 -1 osd.2 0 log_to_monitors true 2024-09-15T21:14:17.680 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:17.681 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:17.681 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:14:17.681 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:14:17.681 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:17.682 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:18.132 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:19.134 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:19.134 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:19.134 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:14:19.135 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:14:19.135 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:19.135 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:19.705 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:20.709 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:20.709 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:20.709 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:14:20.710 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:14:20.710 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:20.710 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:21.167 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:22.169 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:22.169 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:22.169 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:14:22.170 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:14:22.170 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:22.170 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:22.611 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:23.612 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:23.612 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:23.613 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:14:23.613 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:14:23.613 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:23.613 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:24.080 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:25.082 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:25.082 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:25.082 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:14:25.083 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:14:25.083 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:25.083 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:25.613 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:26.614 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:26.615 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:26.615 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:14:26.616 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:14:26.616 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:26.616 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:27.150 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:28.152 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:28.152 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:28.152 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:14:28.153 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:14:28.153 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:28.153 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:28.633 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:29.635 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:29.635 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:29.635 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:14:29.636 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:14:29.636 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:29.636 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:30.288 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:31.290 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:31.290 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:31.290 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:14:31.291 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:14:31.291 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:31.291 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:32.031 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:33.032 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:33.032 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:33.033 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:14:33.033 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:14:33.033 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:33.033 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:33.728 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:34.730 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:34.730 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:34.730 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:14:34.731 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:14:34.731 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:34.731 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:35.569 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:36.570 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:36.570 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:36.571 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:14:36.571 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:14:36.571 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:36.571 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:37.346 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:38.348 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:38.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:38.349 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:14:38.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:14:38.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:38.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:39.194 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:40.196 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:40.196 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:40.197 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:14:40.197 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:14:40.197 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:40.197 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:14:40.696 INFO:tasks.workunit.client.0.smithi016.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2988238342,v1:127.0.0.1:6819/2988238342] [v2:127.0.0.1:6820/2988238342,v1:127.0.0.1:6821/2988238342] exists,up eed50a73-30aa-4b6e-9bf9-4af5e2710c0b 2024-09-15T21:14:40.696 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:14:40.697 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:14:40.697 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:14:40.697 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:182: TEST_reweight: ceph osd crush tree 2024-09-15T21:14:41.160 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT (compat) TYPE NAME 2024-09-15T21:14:41.161 INFO:tasks.workunit.client.0.smithi016.stdout:-1 9.00000 root default 2024-09-15T21:14:41.161 INFO:tasks.workunit.client.0.smithi016.stdout:-2 9.00000 5.00000 host HOST 2024-09-15T21:14:41.161 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 2.00000 osd.0 2024-09-15T21:14:41.161 INFO:tasks.workunit.client.0.smithi016.stdout: 1 3.00000 3.00000 osd.1 2024-09-15T21:14:41.161 INFO:tasks.workunit.client.0.smithi016.stdout: 2 3.00000 0 osd.2 2024-09-15T21:14:41.179 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:183: TEST_reweight: ceph osd crush tree 2024-09-15T21:14:41.179 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:183: TEST_reweight: grep host 2024-09-15T21:14:41.179 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:183: TEST_reweight: grep '9.00000 5.00000' 2024-09-15T21:14:41.670 INFO:tasks.workunit.client.0.smithi016.stdout:-2 9.00000 5.00000 host HOST 2024-09-15T21:14:41.671 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:185: TEST_reweight: ceph osd crush reweight osd.2 4 2024-09-15T21:14:42.352 INFO:tasks.workunit.client.0.smithi016.stderr:reweighted item id 2 name 'osd.2' to 4 in crush map 2024-09-15T21:14:42.369 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:186: TEST_reweight: ceph osd crush tree 2024-09-15T21:14:42.882 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT (compat) TYPE NAME 2024-09-15T21:14:42.882 INFO:tasks.workunit.client.0.smithi016.stdout:-1 10.00000 root default 2024-09-15T21:14:42.882 INFO:tasks.workunit.client.0.smithi016.stdout:-2 10.00000 5.00000 host HOST 2024-09-15T21:14:42.882 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 2.00000 osd.0 2024-09-15T21:14:42.882 INFO:tasks.workunit.client.0.smithi016.stdout: 1 3.00000 3.00000 osd.1 2024-09-15T21:14:42.882 INFO:tasks.workunit.client.0.smithi016.stdout: 2 4.00000 0 osd.2 2024-09-15T21:14:42.900 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:187: TEST_reweight: ceph osd crush tree 2024-09-15T21:14:42.901 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:187: TEST_reweight: grep host 2024-09-15T21:14:42.901 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:187: TEST_reweight: grep '10.00000 5.00000' 2024-09-15T21:14:43.371 INFO:tasks.workunit.client.0.smithi016.stdout:-2 10.00000 5.00000 host HOST 2024-09-15T21:14:43.372 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:189: TEST_reweight: ceph osd crush weight-set reweight-compat osd.2 4 2024-09-15T21:14:44.170 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:190: TEST_reweight: ceph osd crush tree 2024-09-15T21:14:44.663 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT (compat) TYPE NAME 2024-09-15T21:14:44.663 INFO:tasks.workunit.client.0.smithi016.stdout:-1 10.00000 root default 2024-09-15T21:14:44.663 INFO:tasks.workunit.client.0.smithi016.stdout:-2 10.00000 9.00000 host HOST 2024-09-15T21:14:44.663 INFO:tasks.workunit.client.0.smithi016.stdout: 0 3.00000 2.00000 osd.0 2024-09-15T21:14:44.663 INFO:tasks.workunit.client.0.smithi016.stdout: 1 3.00000 3.00000 osd.1 2024-09-15T21:14:44.664 INFO:tasks.workunit.client.0.smithi016.stdout: 2 4.00000 4.00000 osd.2 2024-09-15T21:14:44.679 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:191: TEST_reweight: ceph osd crush tree 2024-09-15T21:14:44.679 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:191: TEST_reweight: grep host 2024-09-15T21:14:44.679 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:191: TEST_reweight: grep '10.00000 9.00000' 2024-09-15T21:14:45.247 INFO:tasks.workunit.client.0.smithi016.stdout:-2 10.00000 9.00000 host HOST 2024-09-15T21:14:45.248 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh:41: run: teardown td/crush-choose-args 2024-09-15T21:14:45.248 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-choose-args 2024-09-15T21:14:45.248 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:14:45.248 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-choose-args KILL 2024-09-15T21:14:45.248 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:14:45.248 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:14:45.249 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:14:45.249 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:14:45.249 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:14:45.360 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:14:45.360 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:14:45.360 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:14:45.361 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:14:45.362 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:14:45.362 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:14:45.362 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:14:45.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:14:45.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:14:45.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:14:45.363 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:14:45.364 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:14:45.365 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:14:45.365 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-choose-args 2024-09-15T21:14:45.378 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:14:45.379 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:14:45.379 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:14:45.379 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.21400 2024-09-15T21:14:45.380 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:14:45.380 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:14:45.380 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2322: main: code=0 2024-09-15T21:14:45.380 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2326: main: teardown td/crush-choose-args 0 2024-09-15T21:14:45.381 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-choose-args 2024-09-15T21:14:45.381 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2024-09-15T21:14:45.381 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-choose-args KILL 2024-09-15T21:14:45.381 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:14:45.381 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:14:45.381 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:14:45.381 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:14:45.381 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:14:45.382 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:14:45.382 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:14:45.383 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:14:45.383 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:14:45.384 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:14:45.384 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:14:45.384 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:14:45.385 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:14:45.385 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:14:45.386 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:14:45.386 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:14:45.386 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:14:45.387 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2024-09-15T21:14:45.387 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-choose-args 2024-09-15T21:14:45.389 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:14:45.389 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:14:45.389 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.21400 2024-09-15T21:14:45.389 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.21400 2024-09-15T21:14:45.390 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:14:45.390 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:14:45.390 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2327: main: return 0 2024-09-15T21:14:45.391 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T21:14:45.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T21:14:45.442 INFO:tasks.workunit:Running workunit crush/crush-classes.sh... 2024-09-15T21:14:45.442 DEBUG:teuthology.orchestra.run.smithi016:workunit test crush/crush-classes.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7498493279982dfe87f00616198a5967475f6169 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh 2024-09-15T21:14:45.495 INFO:tasks.workunit.client.0.smithi016.stderr:stty: 'standard input': Inappropriate ioctl for device 2024-09-15T21:14:45.502 INFO:tasks.workunit.client.0.smithi016.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2024-09-15T21:14:45.502 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2315: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-09-15T21:14:45.502 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2315: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-09-15T21:14:45.502 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2316: main: export PYTHONWARNINGS=ignore 2024-09-15T21:14:45.502 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2316: main: PYTHONWARNINGS=ignore 2024-09-15T21:14:45.502 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2317: main: export CEPH_CONF=/dev/null 2024-09-15T21:14:45.502 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2317: main: CEPH_CONF=/dev/null 2024-09-15T21:14:45.502 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2318: main: unset CEPH_ARGS 2024-09-15T21:14:45.502 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2320: main: local code 2024-09-15T21:14:45.503 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2321: main: run td/crush-classes 2024-09-15T21:14:45.503 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:21: run: local dir=td/crush-classes 2024-09-15T21:14:45.503 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:22: run: shift 2024-09-15T21:14:45.503 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:24: run: export CEPH_MON=127.0.0.1:7130 2024-09-15T21:14:45.503 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:24: run: CEPH_MON=127.0.0.1:7130 2024-09-15T21:14:45.503 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:25: run: export CEPH_ARGS 2024-09-15T21:14:45.503 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:26: run: uuidgen 2024-09-15T21:14:45.504 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:26: run: CEPH_ARGS+='--fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none ' 2024-09-15T21:14:45.504 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:27: run: CEPH_ARGS+='--mon-host=127.0.0.1:7130 ' 2024-09-15T21:14:45.504 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:31: run: CEPH_ARGS+='--osd-class-update-on-start=false ' 2024-09-15T21:14:45.504 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:33: run: set 2024-09-15T21:14:45.504 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:33: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2024-09-15T21:14:45.506 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:33: run: local 'funcs=TEST_classes 2024-09-15T21:14:45.506 INFO:tasks.workunit.client.0.smithi016.stderr:TEST_mon_classes 2024-09-15T21:14:45.506 INFO:tasks.workunit.client.0.smithi016.stderr:TEST_reweight_vs_classes 2024-09-15T21:14:45.507 INFO:tasks.workunit.client.0.smithi016.stderr:TEST_set_device_class' 2024-09-15T21:14:45.507 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:34: run: for func in $funcs 2024-09-15T21:14:45.507 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:35: run: setup td/crush-classes 2024-09-15T21:14:45.507 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/crush-classes 2024-09-15T21:14:45.507 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/crush-classes 2024-09-15T21:14:45.507 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-classes 2024-09-15T21:14:45.507 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:14:45.507 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-classes KILL 2024-09-15T21:14:45.507 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:14:45.507 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:14:45.507 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:14:45.507 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:14:45.507 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:14:45.508 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:14:45.508 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:14:45.509 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:14:45.509 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:14:45.510 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:14:45.510 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:14:45.510 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:14:45.511 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:14:45.511 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:14:45.511 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:14:45.512 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:14:45.513 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:14:45.513 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:14:45.514 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-classes 2024-09-15T21:14:45.515 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:14:45.515 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:14:45.515 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:14:45.515 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39842 2024-09-15T21:14:45.516 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:14:45.516 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:14:45.516 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/crush-classes 2024-09-15T21:14:45.517 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2024-09-15T21:14:45.518 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:14:45.518 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:14:45.518 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.39842 2024-09-15T21:14:45.519 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2024-09-15T21:14:45.520 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2024-09-15T21:14:45.520 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2024-09-15T21:14:45.520 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2024-09-15T21:14:45.521 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/crush-classes 1' TERM HUP INT 2024-09-15T21:14:45.521 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:36: run: TEST_classes td/crush-classes 2024-09-15T21:14:45.521 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:89: TEST_classes: local dir=td/crush-classes 2024-09-15T21:14:45.521 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:91: TEST_classes: run_mon td/crush-classes a 2024-09-15T21:14:45.521 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/crush-classes 2024-09-15T21:14:45.521 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2024-09-15T21:14:45.521 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2024-09-15T21:14:45.521 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2024-09-15T21:14:45.521 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/crush-classes/a 2024-09-15T21:14:45.521 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/crush-classes/a --run-dir=td/crush-classes 2024-09-15T21:14:45.989 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2024-09-15T21:14:45.989 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:14:45.990 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:14:45.990 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:14:45.990 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:14:45.990 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:14:45.990 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:14:45.991 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/crush-classes/a '--log-file=td/crush-classes/$name.log' '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --mon-cluster-log-file=td/crush-classes/log --run-dir=td/crush-classes '--pid-file=td/crush-classes/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2024-09-15T21:14:46.201 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2024-09-15T21:14:46.202 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2024-09-15T21:14:46.202 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:14:46.202 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:14:46.203 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2024-09-15T21:14:46.203 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2024-09-15T21:14:46.203 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:14:46.203 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:14:46.203 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:14:46.204 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:14:46.204 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:14:46.204 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:14:46.205 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39842/ceph-mon.a.asok 2024-09-15T21:14:46.205 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:14:46.205 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39842/ceph-mon.a.asok config get fsid 2024-09-15T21:14:46.321 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2024-09-15T21:14:46.321 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:14:46.321 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:14:46.321 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2024-09-15T21:14:46.321 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2024-09-15T21:14:46.322 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:14:46.322 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:14:46.322 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:14:46.322 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:14:46.323 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:14:46.323 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:14:46.323 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39842/ceph-mon.a.asok 2024-09-15T21:14:46.323 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:14:46.323 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39842/ceph-mon.a.asok config get mon_host 2024-09-15T21:14:46.429 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:92: TEST_classes: run_osd td/crush-classes 0 2024-09-15T21:14:46.429 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-classes 2024-09-15T21:14:46.429 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:14:46.430 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2024-09-15T21:14:46.430 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:14:46.430 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-classes/0 2024-09-15T21:14:46.430 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false ' 2024-09-15T21:14:46.430 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:14:46.430 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:14:46.430 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:14:46.430 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-classes/0' 2024-09-15T21:14:46.430 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-classes/0/journal' 2024-09-15T21:14:46.430 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:14:46.430 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:14:46.430 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-classes' 2024-09-15T21:14:46.431 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:14:46.431 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:14:46.431 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:14:46.432 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:14:46.432 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:14:46.432 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:14:46.432 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:14:46.432 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:14:46.432 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:14:46.432 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:14:46.432 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:14:46.432 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-classes/$name.log' 2024-09-15T21:14:46.432 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-classes/$name.pid' 2024-09-15T21:14:46.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:14:46.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:14:46.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:14:46.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:14:46.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:14:46.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:14:46.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-classes/0 2024-09-15T21:14:46.434 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:14:46.434 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5506a668-9d82-4f04-9df8-aaa0d29abc5c 2024-09-15T21:14:46.434 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5506a668-9d82-4f04-9df8-aaa0d29abc5c' 2024-09-15T21:14:46.435 INFO:tasks.workunit.client.0.smithi016.stdout:add osd0 5506a668-9d82-4f04-9df8-aaa0d29abc5c 2024-09-15T21:14:46.435 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:14:46.458 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBGTudmGWBVGxAAz6QNXLUpNZnxOJ9s0TkSFw== 2024-09-15T21:14:46.458 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBGTudmGWBVGxAAz6QNXLUpNZnxOJ9s0TkSFw=="}' 2024-09-15T21:14:46.459 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5506a668-9d82-4f04-9df8-aaa0d29abc5c -i td/crush-classes/0/new.json 2024-09-15T21:14:47.362 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:14:47.378 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-classes/0/new.json 2024-09-15T21:14:47.380 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/0 --osd-journal=td/crush-classes/0/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBGTudmGWBVGxAAz6QNXLUpNZnxOJ9s0TkSFw== --osd-uuid 5506a668-9d82-4f04-9df8-aaa0d29abc5c 2024-09-15T21:14:47.416 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:47.412+0000 7f178f3a97c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:14:47.430 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:47.428+0000 7f178f3a97c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:14:47.431 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:47.428+0000 7f178f3a97c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:14:47.432 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:47.428+0000 7f178f3a97c0 -1 bdev(0x5568e30fbc00 td/crush-classes/0/block) open stat got: (1) Operation not permitted 2024-09-15T21:14:47.432 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:47.428+0000 7f178f3a97c0 -1 bluestore(td/crush-classes/0) _read_fsid unparsable uuid 2024-09-15T21:14:53.288 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-classes/0/keyring 2024-09-15T21:14:53.288 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:14:53.289 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd0 key to auth repository 2024-09-15T21:14:53.289 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2024-09-15T21:14:53.289 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-classes/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:14:53.549 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.0 2024-09-15T21:14:53.550 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2024-09-15T21:14:53.550 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:14:53.550 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/0 --osd-journal=td/crush-classes/0/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:14:53.550 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:14:53.550 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:14:53.584 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:53.584+0000 7f3ca4ee37c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:14:53.587 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:53.584+0000 7f3ca4ee37c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:14:53.588 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:53.588+0000 7f3ca4ee37c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:14:53.790 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2024-09-15T21:14:53.790 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:14:53.790 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2024-09-15T21:14:53.790 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:14:53.790 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:14:53.791 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:53.791 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:14:53.791 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:14:53.791 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:53.791 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:14:54.037 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:55.038 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:55.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:55.039 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:14:55.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:14:55.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:55.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:14:55.058 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:55.056+0000 7f3ca4ee37c0 -1 Falling back to public interface 2024-09-15T21:14:55.280 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:56.281 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:56.282 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:56.282 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:14:56.282 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:14:56.283 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:56.283 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:14:56.521 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:57.523 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:57.523 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:57.523 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:14:57.523 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:14:57.524 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:57.524 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:14:57.664 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:14:57.660+0000 7f3ca4ee37c0 -1 osd.0 0 log_to_monitors true 2024-09-15T21:14:57.924 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:14:58.926 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:14:58.926 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:14:58.926 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:14:58.926 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:14:58.927 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:14:58.927 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:14:59.172 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:00.173 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:00.174 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:00.174 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:15:00.174 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:15:00.175 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:00.175 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:00.419 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:01.421 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:01.421 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:15:01.422 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:01.422 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:15:01.422 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:01.422 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:01.659 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:02.661 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:02.720 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:02.720 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:15:02.720 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:15:02.721 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:02.721 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:02.905 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:03.907 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:03.907 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:03.907 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:15:03.908 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:15:03.908 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:03.908 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:04.152 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:05.154 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:05.154 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:05.154 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:15:05.154 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:15:05.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:05.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:05.398 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:06.400 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:06.400 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:06.400 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:15:06.401 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:15:06.401 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:06.401 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:06.652 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:07.654 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:07.654 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:07.654 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:15:07.655 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:15:07.655 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:07.656 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:07.901 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:08.903 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:08.903 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:08.903 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:15:08.903 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:15:08.904 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:08.904 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:09.150 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:10.152 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:10.152 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:10.152 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:15:10.153 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:15:10.153 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:10.153 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:10.398 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:11.400 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:11.400 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:11.400 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:15:11.401 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:15:11.401 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:11.401 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:11.645 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:12.647 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:12.647 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:12.647 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:15:12.647 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:15:12.648 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:12.648 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:12.895 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:13.897 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:13.897 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:13.897 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:15:13.898 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:15:13.898 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:13.898 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:14.142 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:15.144 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:15.144 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:15.144 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:15:15.145 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:15:15.145 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:15.146 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:15.389 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:16.391 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:16.391 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:16.392 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 18 2024-09-15T21:15:16.392 INFO:tasks.workunit.client.0.smithi016.stdout:18 2024-09-15T21:15:16.393 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:16.393 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:16.637 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:17.638 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:17.639 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:17.639 INFO:tasks.workunit.client.0.smithi016.stdout:19 2024-09-15T21:15:17.640 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 19 2024-09-15T21:15:17.640 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:17.640 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:17.887 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:18.888 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:18.889 INFO:tasks.workunit.client.0.smithi016.stdout:20 2024-09-15T21:15:18.889 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:18.890 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 20 2024-09-15T21:15:18.890 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:18.890 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:15:19.151 INFO:tasks.workunit.client.0.smithi016.stdout:osd.0 up in weight 1 up_from 4 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/280858990,v1:127.0.0.1:6801/280858990] [v2:127.0.0.1:6802/280858990,v1:127.0.0.1:6803/280858990] exists,up 5506a668-9d82-4f04-9df8-aaa0d29abc5c 2024-09-15T21:15:19.154 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:15:19.154 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:15:19.154 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:15:19.154 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:93: TEST_classes: run_osd td/crush-classes 1 2024-09-15T21:15:19.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-classes 2024-09-15T21:15:19.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:15:19.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2024-09-15T21:15:19.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:15:19.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-classes/1 2024-09-15T21:15:19.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false ' 2024-09-15T21:15:19.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:15:19.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:15:19.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:15:19.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-classes/1' 2024-09-15T21:15:19.155 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-classes/1/journal' 2024-09-15T21:15:19.156 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:15:19.156 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:15:19.156 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-classes' 2024-09-15T21:15:19.156 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:15:19.156 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:15:19.156 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:15:19.157 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:15:19.157 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:15:19.157 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:15:19.157 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:15:19.157 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:15:19.157 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:15:19.157 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:15:19.157 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:15:19.157 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-classes/$name.log' 2024-09-15T21:15:19.157 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-classes/$name.pid' 2024-09-15T21:15:19.157 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:15:19.157 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:15:19.158 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:15:19.158 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:15:19.158 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:15:19.158 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:15:19.158 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-classes/1 2024-09-15T21:15:19.158 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:15:19.159 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6b949b10-b113-4483-832b-7ccca0ac10fc 2024-09-15T21:15:19.159 INFO:tasks.workunit.client.0.smithi016.stdout:add osd1 6b949b10-b113-4483-832b-7ccca0ac10fc 2024-09-15T21:15:19.159 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 6b949b10-b113-4483-832b-7ccca0ac10fc' 2024-09-15T21:15:19.159 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:15:19.181 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBnTudmRNnMChAAtb4Grhg706P0M+kdT28ISw== 2024-09-15T21:15:19.181 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBnTudmRNnMChAAtb4Grhg706P0M+kdT28ISw=="}' 2024-09-15T21:15:19.181 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6b949b10-b113-4483-832b-7ccca0ac10fc -i td/crush-classes/1/new.json 2024-09-15T21:15:19.497 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:15:19.515 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-classes/1/new.json 2024-09-15T21:15:19.517 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/1 --osd-journal=td/crush-classes/1/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBnTudmRNnMChAAtb4Grhg706P0M+kdT28ISw== --osd-uuid 6b949b10-b113-4483-832b-7ccca0ac10fc 2024-09-15T21:15:19.554 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:19.552+0000 7fcfba5767c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:15:19.583 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:19.580+0000 7fcfba5767c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:15:19.585 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:19.584+0000 7fcfba5767c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:15:19.586 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:19.584+0000 7fcfba5767c0 -1 bdev(0x556c63827c00 td/crush-classes/1/block) open stat got: (1) Operation not permitted 2024-09-15T21:15:19.586 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:19.584+0000 7fcfba5767c0 -1 bluestore(td/crush-classes/1) _read_fsid unparsable uuid 2024-09-15T21:15:25.254 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-classes/1/keyring 2024-09-15T21:15:25.254 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:15:25.256 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2024-09-15T21:15:25.256 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd1 key to auth repository 2024-09-15T21:15:25.256 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-classes/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:15:25.521 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.1 2024-09-15T21:15:25.521 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2024-09-15T21:15:25.522 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/1 --osd-journal=td/crush-classes/1/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:15:25.522 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:15:25.522 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:15:25.522 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:15:25.554 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:25.552+0000 7f249d56a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:15:25.557 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:25.556+0000 7f249d56a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:15:25.558 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:25.556+0000 7f249d56a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:15:25.761 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2024-09-15T21:15:25.761 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:15:25.761 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2024-09-15T21:15:25.761 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:15:25.762 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:15:25.762 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:25.762 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:15:25.762 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:15:25.762 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:25.763 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:26.000 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:26.627 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:26.624+0000 7f249d56a7c0 -1 Falling back to public interface 2024-09-15T21:15:27.002 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:27.002 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:27.002 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:15:27.003 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:15:27.003 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:27.003 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:27.245 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:28.247 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:28.247 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:28.247 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:15:28.248 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:15:28.248 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:28.248 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:28.492 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:29.339 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:29.336+0000 7f249d56a7c0 -1 osd.1 0 log_to_monitors true 2024-09-15T21:15:29.493 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:29.493 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:29.493 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:15:29.494 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:15:29.494 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:29.494 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:29.870 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:30.872 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:30.872 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:30.872 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:15:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:15:30.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:30.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:31.117 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:32.119 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:32.119 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:32.119 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:15:32.120 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:15:32.120 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:32.120 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:32.364 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:33.366 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:33.366 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:33.366 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:15:33.367 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:15:33.367 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:33.367 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:33.607 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:34.609 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:34.609 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:34.609 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:15:34.610 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:15:34.610 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:34.610 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:34.853 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:35.854 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:35.854 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:35.855 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:15:35.855 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:15:35.855 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:35.855 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:36.099 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:37.101 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:37.101 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:37.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:15:37.102 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:15:37.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:37.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:37.348 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:38.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:38.349 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:15:38.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:38.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:15:38.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:38.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:38.598 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:39.600 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:39.601 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:39.601 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:15:39.601 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:15:39.601 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:39.601 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:39.846 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:40.848 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:40.848 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:40.848 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:15:40.849 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:15:40.849 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:40.849 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:41.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:42.093 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:42.093 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:42.093 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:15:42.094 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:15:42.094 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:42.094 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:42.338 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:43.339 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:43.340 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:43.340 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:15:43.340 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:15:43.340 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:43.340 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:43.584 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:44.586 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:44.586 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:44.586 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:15:44.587 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:15:44.587 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:44.587 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:44.833 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:45.835 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:45.835 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:45.835 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:15:45.836 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:15:45.836 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:45.836 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:46.075 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:47.077 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:47.077 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:47.078 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:15:47.078 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:15:47.079 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:47.079 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:47.323 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:48.325 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:48.325 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:48.325 INFO:tasks.workunit.client.0.smithi016.stdout:18 2024-09-15T21:15:48.326 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 18 2024-09-15T21:15:48.326 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:48.326 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:48.568 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:49.569 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:49.570 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:49.570 INFO:tasks.workunit.client.0.smithi016.stdout:19 2024-09-15T21:15:49.571 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 19 2024-09-15T21:15:49.571 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:49.572 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:15:49.815 INFO:tasks.workunit.client.0.smithi016.stdout:osd.1 up in weight 1 up_from 8 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6808/2243216895,v1:127.0.0.1:6809/2243216895] [v2:127.0.0.1:6810/2243216895,v1:127.0.0.1:6811/2243216895] exists,up 6b949b10-b113-4483-832b-7ccca0ac10fc 2024-09-15T21:15:49.817 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:15:49.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:15:49.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:15:49.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:94: TEST_classes: run_osd td/crush-classes 2 2024-09-15T21:15:49.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-classes 2024-09-15T21:15:49.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:15:49.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2024-09-15T21:15:49.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:15:49.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-classes/2 2024-09-15T21:15:49.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false ' 2024-09-15T21:15:49.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:15:49.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:15:49.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:15:49.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-classes/2' 2024-09-15T21:15:49.819 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-classes/2/journal' 2024-09-15T21:15:49.819 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:15:49.819 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:15:49.819 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-classes' 2024-09-15T21:15:49.819 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:15:49.819 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:15:49.819 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:15:49.819 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:15:49.819 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:15:49.819 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:15:49.819 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:15:49.820 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:15:49.820 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:15:49.820 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:15:49.820 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:15:49.821 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-classes/$name.log' 2024-09-15T21:15:49.821 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-classes/$name.pid' 2024-09-15T21:15:49.821 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:15:49.821 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:15:49.821 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:15:49.821 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:15:49.821 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:15:49.821 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:15:49.821 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-classes/2 2024-09-15T21:15:49.821 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:15:49.822 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=49b16c2e-1c85-4bc0-9ab4-412768bacd40 2024-09-15T21:15:49.822 INFO:tasks.workunit.client.0.smithi016.stdout:add osd2 49b16c2e-1c85-4bc0-9ab4-412768bacd40 2024-09-15T21:15:49.822 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 49b16c2e-1c85-4bc0-9ab4-412768bacd40' 2024-09-15T21:15:49.822 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:15:49.840 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCFTudmef0eMhAA/glJRIP186i4cu+lxKpszA== 2024-09-15T21:15:49.840 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCFTudmef0eMhAA/glJRIP186i4cu+lxKpszA=="}' 2024-09-15T21:15:49.840 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 49b16c2e-1c85-4bc0-9ab4-412768bacd40 -i td/crush-classes/2/new.json 2024-09-15T21:15:50.169 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:15:50.186 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-classes/2/new.json 2024-09-15T21:15:50.188 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/2 --osd-journal=td/crush-classes/2/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCFTudmef0eMhAA/glJRIP186i4cu+lxKpszA== --osd-uuid 49b16c2e-1c85-4bc0-9ab4-412768bacd40 2024-09-15T21:15:50.224 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:50.224+0000 7fe67025a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:15:50.279 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:50.276+0000 7fe67025a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:15:50.281 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:50.280+0000 7fe67025a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:15:50.281 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:50.280+0000 7fe67025a7c0 -1 bdev(0x564998875c00 td/crush-classes/2/block) open stat got: (1) Operation not permitted 2024-09-15T21:15:50.281 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:50.280+0000 7fe67025a7c0 -1 bluestore(td/crush-classes/2) _read_fsid unparsable uuid 2024-09-15T21:15:56.149 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-classes/2/keyring 2024-09-15T21:15:56.150 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:15:56.151 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd2 key to auth repository 2024-09-15T21:15:56.151 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2024-09-15T21:15:56.152 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-classes/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:15:56.409 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.2 2024-09-15T21:15:56.410 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2024-09-15T21:15:56.410 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/2 --osd-journal=td/crush-classes/2/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:15:56.410 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:15:56.410 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:15:56.410 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:15:56.444 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:56.440+0000 7f78953497c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:15:56.446 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:56.444+0000 7f78953497c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:15:56.448 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:56.448+0000 7f78953497c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:15:56.651 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2024-09-15T21:15:56.651 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:15:56.652 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2024-09-15T21:15:56.652 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:15:56.652 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:15:56.652 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:56.652 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:15:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:15:56.653 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:15:56.653 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:56.880 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:57.495 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:15:57.491+0000 7f78953497c0 -1 Falling back to public interface 2024-09-15T21:15:57.881 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:57.881 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:57.882 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:15:57.882 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:15:57.882 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:57.882 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:15:58.122 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:15:59.124 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:15:59.124 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:15:59.124 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:15:59.125 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:15:59.125 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:15:59.125 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:15:59.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:00.364 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:00.364 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:00.365 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:16:00.365 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:16:00.365 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:00.365 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:00.371 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:16:00.367+0000 7f78953497c0 -1 osd.2 0 log_to_monitors true 2024-09-15T21:16:00.871 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:01.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:01.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:01.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:16:01.873 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:16:01.874 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:01.874 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:02.119 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:03.120 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:03.121 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:03.121 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:16:03.121 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:16:03.121 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:03.121 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:03.366 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:04.368 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:04.368 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:04.368 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:16:04.369 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:16:04.369 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:04.369 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:04.608 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:05.610 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:05.610 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:05.610 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:16:05.611 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:16:05.611 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:05.611 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:05.855 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:06.856 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:06.856 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:06.857 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:16:06.857 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:16:06.857 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:06.857 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:07.103 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:08.104 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:08.105 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:08.105 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:16:08.105 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:16:08.105 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:08.106 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:08.351 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:09.352 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:09.352 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:09.353 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:16:09.353 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:16:09.353 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:09.353 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:09.602 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:10.604 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:10.604 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:10.604 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:16:10.605 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:16:10.605 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:10.605 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:10.848 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:11.849 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:11.849 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:11.850 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:16:11.850 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:16:11.850 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:11.850 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:12.092 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:13.094 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:13.094 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:13.094 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:16:13.094 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:16:13.095 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:13.095 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:13.339 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:14.341 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:14.341 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:14.341 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:16:14.341 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:16:14.341 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:14.342 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:14.590 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:15.592 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:15.592 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:15.592 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:16:15.593 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:16:15.593 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:15.593 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:15.841 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:16.843 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:16.843 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:16.843 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:16:16.844 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:16:16.844 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:16.844 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:17.087 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:18.089 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:18.089 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:18.089 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:16:18.089 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:16:18.089 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:18.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:18.335 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:19.336 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:19.336 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:19.337 INFO:tasks.workunit.client.0.smithi016.stdout:18 2024-09-15T21:16:19.337 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 18 2024-09-15T21:16:19.337 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:19.337 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:19.577 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:20.578 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:20.579 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:20.579 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 19 2024-09-15T21:16:20.579 INFO:tasks.workunit.client.0.smithi016.stdout:19 2024-09-15T21:16:20.579 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:20.579 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:20.871 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:21.872 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:21.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:21.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 20 2024-09-15T21:16:21.873 INFO:tasks.workunit.client.0.smithi016.stdout:20 2024-09-15T21:16:21.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:21.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:16:22.117 INFO:tasks.workunit.client.0.smithi016.stdout:osd.2 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6816/2116971322,v1:127.0.0.1:6817/2116971322] [v2:127.0.0.1:6818/2116971322,v1:127.0.0.1:6819/2116971322] exists,up 49b16c2e-1c85-4bc0-9ab4-412768bacd40 2024-09-15T21:16:22.118 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:16:22.118 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:16:22.118 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:16:22.118 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:95: TEST_classes: create_rbd_pool 2024-09-15T21:16:22.118 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2024-09-15T21:16:22.346 INFO:tasks.workunit.client.0.smithi016.stderr:pool 'rbd' does not exist 2024-09-15T21:16:22.361 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2024-09-15T21:16:22.361 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2024-09-15T21:16:22.745 INFO:tasks.workunit.client.0.smithi016.stderr:pool 'rbd' already exists 2024-09-15T21:16:22.766 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2024-09-15T21:16:23.768 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2024-09-15T21:16:25.502 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:97: TEST_classes: get_osds_up rbd SOMETHING 2024-09-15T21:16:25.502 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:51: get_osds_up: local poolname=rbd 2024-09-15T21:16:25.503 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:52: get_osds_up: local objectname=SOMETHING 2024-09-15T21:16:25.503 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: ceph --format xml osd map rbd SOMETHING 2024-09-15T21:16:25.503 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: xmlstarlet sel -t -m //up/osd -v . -o ' ' 2024-09-15T21:16:25.786 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: local 'osds=1 2 0 ' 2024-09-15T21:16:25.786 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:57: get_osds_up: echo 1 2 0 2024-09-15T21:16:25.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:97: TEST_classes: test '1 2 0' == '1 2 0' 2024-09-15T21:16:25.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:98: TEST_classes: add_something td/crush-classes SOMETHING 2024-09-15T21:16:25.788 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:42: add_something: local dir=td/crush-classes 2024-09-15T21:16:25.788 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:43: add_something: local obj=SOMETHING 2024-09-15T21:16:25.788 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:45: add_something: local payload=ABCDEF 2024-09-15T21:16:25.788 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:46: add_something: echo ABCDEF 2024-09-15T21:16:25.788 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:47: add_something: rados --pool rbd put SOMETHING td/crush-classes/ORIGINAL 2024-09-15T21:16:25.969 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:104: TEST_classes: ceph osd getcrushmap 2024-09-15T21:16:26.196 INFO:tasks.workunit.client.0.smithi016.stderr:4 2024-09-15T21:16:26.212 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:105: TEST_classes: crushtool -d td/crush-classes/map -o td/crush-classes/map.txt 2024-09-15T21:16:26.234 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:106: TEST_classes: sed -i -e '/device 0 osd.0/s/$/ class ssd/' -e '/step take default/s/$/ class ssd/' td/crush-classes/map.txt 2024-09-15T21:16:26.236 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:110: TEST_classes: crushtool -c td/crush-classes/map.txt -o td/crush-classes/map-new 2024-09-15T21:16:26.254 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:111: TEST_classes: ceph osd setcrushmap -i td/crush-classes/map-new 2024-09-15T21:16:26.943 INFO:tasks.workunit.client.0.smithi016.stderr:6 2024-09-15T21:16:26.968 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:117: TEST_classes: ok=false 2024-09-15T21:16:26.968 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:118: TEST_classes: for delay in 2 4 8 16 32 64 128 256 2024-09-15T21:16:26.969 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:119: TEST_classes: get_osds_up rbd SOMETHING_ELSE 2024-09-15T21:16:26.969 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:51: get_osds_up: local poolname=rbd 2024-09-15T21:16:26.969 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:52: get_osds_up: local objectname=SOMETHING_ELSE 2024-09-15T21:16:26.969 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: ceph --format xml osd map rbd SOMETHING_ELSE 2024-09-15T21:16:26.970 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: xmlstarlet sel -t -m //up/osd -v . -o ' ' 2024-09-15T21:16:27.349 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: local 'osds=0 ' 2024-09-15T21:16:27.349 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:57: get_osds_up: echo 0 2024-09-15T21:16:27.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:119: TEST_classes: test 0 == 0 2024-09-15T21:16:27.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:120: TEST_classes: ok=true 2024-09-15T21:16:27.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:121: TEST_classes: break 2024-09-15T21:16:27.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:127: TEST_classes: true 2024-09-15T21:16:27.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:132: TEST_classes: add_something td/crush-classes SOMETHING_ELSE 2024-09-15T21:16:27.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:42: add_something: local dir=td/crush-classes 2024-09-15T21:16:27.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:43: add_something: local obj=SOMETHING_ELSE 2024-09-15T21:16:27.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:45: add_something: local payload=ABCDEF 2024-09-15T21:16:27.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:46: add_something: echo ABCDEF 2024-09-15T21:16:27.350 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:47: add_something: rados --pool rbd put SOMETHING_ELSE td/crush-classes/ORIGINAL 2024-09-15T21:16:28.407 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:138: TEST_classes: ceph osd crush dump 2024-09-15T21:16:28.407 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:138: TEST_classes: grep -q '~ssd' 2024-09-15T21:16:28.652 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:37: run: teardown td/crush-classes 2024-09-15T21:16:28.652 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-classes 2024-09-15T21:16:28.652 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:16:28.653 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-classes KILL 2024-09-15T21:16:28.653 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:16:28.653 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:16:28.654 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:16:28.654 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:16:28.654 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:16:28.763 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:16:28.764 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:16:28.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:16:28.764 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:16:28.765 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:16:28.765 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:16:28.766 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:16:28.766 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:16:28.766 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:16:28.767 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:16:28.767 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:16:28.767 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:16:28.768 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:16:28.768 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-classes 2024-09-15T21:16:28.784 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:16:28.784 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:16:28.784 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:16:28.784 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39842 2024-09-15T21:16:28.786 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:16:28.786 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:16:28.786 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:34: run: for func in $funcs 2024-09-15T21:16:28.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:35: run: setup td/crush-classes 2024-09-15T21:16:28.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/crush-classes 2024-09-15T21:16:28.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/crush-classes 2024-09-15T21:16:28.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-classes 2024-09-15T21:16:28.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:16:28.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-classes KILL 2024-09-15T21:16:28.787 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:16:28.787 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:16:28.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:16:28.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:16:28.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:16:28.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:16:28.787 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:16:28.788 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:16:28.788 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:16:28.788 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:16:28.789 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:16:28.789 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:16:28.789 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:16:28.789 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:16:28.789 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:16:28.789 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:16:28.789 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:16:28.791 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:16:28.791 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-classes 2024-09-15T21:16:28.792 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:16:28.792 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:16:28.792 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:16:28.792 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39842 2024-09-15T21:16:28.793 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:16:28.793 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:16:28.793 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/crush-classes 2024-09-15T21:16:28.795 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2024-09-15T21:16:28.795 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:16:28.795 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:16:28.795 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.39842 2024-09-15T21:16:28.797 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2024-09-15T21:16:28.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2024-09-15T21:16:28.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2024-09-15T21:16:28.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/crush-classes 1' TERM HUP INT 2024-09-15T21:16:28.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:36: run: TEST_mon_classes td/crush-classes 2024-09-15T21:16:28.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:167: TEST_mon_classes: local dir=td/crush-classes 2024-09-15T21:16:28.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:169: TEST_mon_classes: run_mon td/crush-classes a 2024-09-15T21:16:28.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/crush-classes 2024-09-15T21:16:28.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2024-09-15T21:16:28.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2024-09-15T21:16:28.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2024-09-15T21:16:28.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/crush-classes/a 2024-09-15T21:16:28.798 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/crush-classes/a --run-dir=td/crush-classes 2024-09-15T21:16:29.254 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2024-09-15T21:16:29.255 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:16:29.255 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:16:29.255 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:16:29.255 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:16:29.255 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:16:29.256 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:16:29.256 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/crush-classes/a '--log-file=td/crush-classes/$name.log' '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --mon-cluster-log-file=td/crush-classes/log --run-dir=td/crush-classes '--pid-file=td/crush-classes/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2024-09-15T21:16:29.459 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2024-09-15T21:16:29.461 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2024-09-15T21:16:29.461 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:16:29.461 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:16:29.461 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2024-09-15T21:16:29.462 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2024-09-15T21:16:29.462 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:16:29.462 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:16:29.462 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:16:29.463 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:16:29.463 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:16:29.463 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:16:29.463 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39842/ceph-mon.a.asok 2024-09-15T21:16:29.463 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:16:29.464 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39842/ceph-mon.a.asok config get fsid 2024-09-15T21:16:29.582 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2024-09-15T21:16:29.582 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:16:29.582 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:16:29.582 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2024-09-15T21:16:29.583 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:16:29.583 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2024-09-15T21:16:29.583 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:16:29.583 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:16:29.583 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:16:29.583 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:16:29.583 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:16:29.584 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39842/ceph-mon.a.asok 2024-09-15T21:16:29.584 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:16:29.584 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39842/ceph-mon.a.asok config get mon_host 2024-09-15T21:16:29.698 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:170: TEST_mon_classes: run_osd td/crush-classes 0 2024-09-15T21:16:29.698 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-classes 2024-09-15T21:16:29.698 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:16:29.698 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2024-09-15T21:16:29.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:16:29.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-classes/0 2024-09-15T21:16:29.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false ' 2024-09-15T21:16:29.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:16:29.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:16:29.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:16:29.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-classes/0' 2024-09-15T21:16:29.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-classes/0/journal' 2024-09-15T21:16:29.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:16:29.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:16:29.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-classes' 2024-09-15T21:16:29.699 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:16:29.699 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:16:29.700 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:16:29.700 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:16:29.700 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:16:29.700 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:16:29.700 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:16:29.700 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:16:29.700 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:16:29.700 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:16:29.700 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:16:29.700 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-classes/$name.log' 2024-09-15T21:16:29.700 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-classes/$name.pid' 2024-09-15T21:16:29.700 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:16:29.700 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:16:29.701 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:16:29.701 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:16:29.701 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:16:29.701 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:16:29.701 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-classes/0 2024-09-15T21:16:29.701 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:16:29.701 INFO:tasks.workunit.client.0.smithi016.stdout:add osd0 b80eed2f-5e8b-4dc1-af19-2cbafcdedc97 2024-09-15T21:16:29.702 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b80eed2f-5e8b-4dc1-af19-2cbafcdedc97 2024-09-15T21:16:29.702 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b80eed2f-5e8b-4dc1-af19-2cbafcdedc97' 2024-09-15T21:16:29.702 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:16:29.724 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCtTudm6bk1KxAAUXFqt+770S6NwHwdSzyjkg== 2024-09-15T21:16:29.724 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCtTudm6bk1KxAAUXFqt+770S6NwHwdSzyjkg=="}' 2024-09-15T21:16:29.724 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b80eed2f-5e8b-4dc1-af19-2cbafcdedc97 -i td/crush-classes/0/new.json 2024-09-15T21:16:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:16:30.679 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-classes/0/new.json 2024-09-15T21:16:30.680 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/0 --osd-journal=td/crush-classes/0/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCtTudm6bk1KxAAUXFqt+770S6NwHwdSzyjkg== --osd-uuid b80eed2f-5e8b-4dc1-af19-2cbafcdedc97 2024-09-15T21:16:30.718 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:16:30.715+0000 7f6472a5b7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:16:30.720 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:16:30.719+0000 7f6472a5b7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:16:30.721 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:16:30.719+0000 7f6472a5b7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:16:30.722 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:16:30.719+0000 7f6472a5b7c0 -1 bdev(0x55dca4841c00 td/crush-classes/0/block) open stat got: (1) Operation not permitted 2024-09-15T21:16:30.722 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:16:30.719+0000 7f6472a5b7c0 -1 bluestore(td/crush-classes/0) _read_fsid unparsable uuid 2024-09-15T21:16:36.543 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-classes/0/keyring 2024-09-15T21:16:36.543 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:16:36.544 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd0 key to auth repository 2024-09-15T21:16:36.545 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2024-09-15T21:16:36.545 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-classes/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:16:36.800 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.0 2024-09-15T21:16:36.801 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2024-09-15T21:16:36.801 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:16:36.801 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/0 --osd-journal=td/crush-classes/0/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:16:36.801 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:16:36.801 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:16:36.834 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:16:36.831+0000 7f1fa7da77c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:16:36.837 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:16:36.835+0000 7f1fa7da77c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:16:36.838 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:16:36.835+0000 7f1fa7da77c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:16:37.032 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2024-09-15T21:16:37.033 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:16:37.033 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2024-09-15T21:16:37.033 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:16:37.033 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:16:37.033 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:16:37.033 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:37.034 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:16:37.034 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:37.034 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:37.261 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:38.091 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:16:38.087+0000 7f1fa7da77c0 -1 Falling back to public interface 2024-09-15T21:16:38.263 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:38.263 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:38.263 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:16:38.263 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:16:38.264 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:38.264 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:38.505 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:39.506 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:39.507 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:39.507 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:16:39.507 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:16:39.507 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:39.507 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:39.749 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:40.710 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:16:40.707+0000 7f1fa7da77c0 -1 osd.0 0 log_to_monitors true 2024-09-15T21:16:40.750 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:40.751 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:16:40.751 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:40.751 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:16:40.751 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:40.751 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:41.218 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:42.220 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:42.220 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:16:42.221 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:42.221 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:16:42.221 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:42.221 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:42.465 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:43.467 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:43.468 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:43.468 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:16:43.468 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:16:43.469 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:43.469 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:43.714 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:44.716 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:44.716 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:44.716 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:16:44.717 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:16:44.717 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:44.717 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:44.959 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:45.960 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:45.960 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:45.960 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:16:45.961 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:16:45.961 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:45.961 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:46.208 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:47.210 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:47.210 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:47.210 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:16:47.211 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:16:47.211 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:47.211 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:47.454 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:48.455 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:48.455 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:48.456 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:16:48.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:16:48.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:48.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:48.702 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:49.705 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:49.705 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:49.705 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:16:49.706 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:16:49.706 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:49.706 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:49.949 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:50.951 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:50.951 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:50.951 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:16:50.952 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:16:50.952 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:50.952 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:51.196 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:52.197 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:52.198 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:52.198 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:16:52.198 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:16:52.198 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:52.199 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:52.443 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:53.444 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:53.445 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:53.445 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:16:53.445 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:16:53.445 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:53.446 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:53.694 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:54.696 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:54.696 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:54.696 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:16:54.697 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:16:54.697 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:54.697 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:54.945 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:55.947 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:55.947 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:55.947 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:16:55.948 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:16:55.948 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:55.948 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:56.194 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:57.195 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:57.195 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:57.196 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:16:57.196 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:16:57.196 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:57.196 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:57.440 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:58.442 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:58.442 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:58.442 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:16:58.443 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:16:58.443 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:58.443 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:58.692 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:16:59.693 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:16:59.693 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:16:59.694 INFO:tasks.workunit.client.0.smithi016.stdout:18 2024-09-15T21:16:59.694 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 18 2024-09-15T21:16:59.694 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:16:59.694 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:16:59.939 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:00.940 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:00.940 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:00.941 INFO:tasks.workunit.client.0.smithi016.stdout:19 2024-09-15T21:17:00.941 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 19 2024-09-15T21:17:00.941 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:00.941 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:17:01.310 INFO:tasks.workunit.client.0.smithi016.stdout:osd.0 up in weight 1 up_from 4 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1298222859,v1:127.0.0.1:6801/1298222859] [v2:127.0.0.1:6802/1298222859,v1:127.0.0.1:6803/1298222859] exists,up b80eed2f-5e8b-4dc1-af19-2cbafcdedc97 2024-09-15T21:17:01.313 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:17:01.313 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:17:01.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:17:01.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:171: TEST_mon_classes: run_osd td/crush-classes 1 2024-09-15T21:17:01.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-classes 2024-09-15T21:17:01.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:17:01.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2024-09-15T21:17:01.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:17:01.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-classes/1 2024-09-15T21:17:01.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false ' 2024-09-15T21:17:01.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:17:01.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:17:01.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:17:01.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-classes/1' 2024-09-15T21:17:01.315 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-classes/1/journal' 2024-09-15T21:17:01.315 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:17:01.315 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:17:01.315 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-classes' 2024-09-15T21:17:01.315 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:17:01.315 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:17:01.315 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:17:01.315 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:17:01.315 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:17:01.315 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:17:01.315 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:17:01.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:17:01.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:17:01.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:17:01.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:17:01.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-classes/$name.log' 2024-09-15T21:17:01.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-classes/$name.pid' 2024-09-15T21:17:01.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:17:01.318 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:17:01.318 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:17:01.318 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:17:01.318 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:17:01.318 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:17:01.318 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-classes/1 2024-09-15T21:17:01.318 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:17:01.318 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1599467e-e997-4c89-90cc-fe35cd0339c0 2024-09-15T21:17:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:add osd1 1599467e-e997-4c89-90cc-fe35cd0339c0 2024-09-15T21:17:01.319 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 1599467e-e997-4c89-90cc-fe35cd0339c0' 2024-09-15T21:17:01.319 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:17:01.336 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDNTudm5ocgFBAAbm52cHd3yPTLdsunOu+ocQ== 2024-09-15T21:17:01.337 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDNTudm5ocgFBAAbm52cHd3yPTLdsunOu+ocQ=="}' 2024-09-15T21:17:01.337 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1599467e-e997-4c89-90cc-fe35cd0339c0 -i td/crush-classes/1/new.json 2024-09-15T21:17:01.670 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:17:01.688 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-classes/1/new.json 2024-09-15T21:17:01.690 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/1 --osd-journal=td/crush-classes/1/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDNTudm5ocgFBAAbm52cHd3yPTLdsunOu+ocQ== --osd-uuid 1599467e-e997-4c89-90cc-fe35cd0339c0 2024-09-15T21:17:01.726 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:01.723+0000 7f86493547c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:17:01.748 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:01.747+0000 7f86493547c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:17:01.749 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:01.747+0000 7f86493547c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:17:01.750 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:01.747+0000 7f86493547c0 -1 bdev(0x560217f31c00 td/crush-classes/1/block) open stat got: (1) Operation not permitted 2024-09-15T21:17:01.750 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:01.747+0000 7f86493547c0 -1 bluestore(td/crush-classes/1) _read_fsid unparsable uuid 2024-09-15T21:17:07.847 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-classes/1/keyring 2024-09-15T21:17:07.847 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:17:07.849 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd1 key to auth repository 2024-09-15T21:17:07.849 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2024-09-15T21:17:07.849 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-classes/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:17:08.107 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.1 2024-09-15T21:17:08.107 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2024-09-15T21:17:08.107 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/1 --osd-journal=td/crush-classes/1/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:17:08.107 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:17:08.107 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:17:08.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:17:08.140 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:08.139+0000 7f43451577c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:17:08.143 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:08.139+0000 7f43451577c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:17:08.144 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:08.143+0000 7f43451577c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:17:08.348 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2024-09-15T21:17:08.348 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:17:08.348 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2024-09-15T21:17:08.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:17:08.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:17:08.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:08.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:17:08.349 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:17:08.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:08.349 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:08.583 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:09.454 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:09.451+0000 7f43451577c0 -1 Falling back to public interface 2024-09-15T21:17:09.585 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:09.585 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:09.585 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:17:09.585 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:17:09.586 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:09.586 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:09.829 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:10.830 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:10.830 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:10.830 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:17:10.831 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:17:10.831 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:10.831 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:11.081 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:12.083 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:12.083 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:12.083 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:17:12.083 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:17:12.084 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:12.084 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:12.121 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:12.119+0000 7f43451577c0 -1 osd.1 0 log_to_monitors true 2024-09-15T21:17:12.430 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:13.431 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:13.432 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:13.432 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:17:13.432 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:17:13.432 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:13.432 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:13.677 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:14.679 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:14.679 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:14.679 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:17:14.680 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:17:14.680 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:14.680 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:14.924 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:15.926 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:15.926 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:15.927 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:17:15.927 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:17:15.927 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:15.927 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:16.168 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:17.170 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:17.170 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:17.170 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:17:17.171 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:17:17.171 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:17.171 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:17.415 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:18.417 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:18.417 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:18.417 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:17:18.418 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:17:18.418 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:18.418 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:18.664 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:19.666 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:19.666 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:19.666 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:17:19.667 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:17:19.667 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:19.667 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:19.911 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:20.913 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:20.913 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:20.914 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:17:20.914 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:17:20.914 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:20.914 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:21.158 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:22.159 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:22.159 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:22.159 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:17:22.160 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:17:22.160 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:22.160 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:22.397 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:23.399 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:23.399 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:23.399 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:17:23.400 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:17:23.400 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:23.400 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:23.640 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:24.642 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:24.642 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:24.642 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:17:24.643 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:17:24.643 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:24.643 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:24.884 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:25.886 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:25.886 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:25.887 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:17:25.887 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:17:25.887 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:25.887 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:26.131 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:27.133 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:27.133 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:27.133 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:17:27.133 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:17:27.134 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:27.134 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:27.371 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:28.373 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:28.373 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:28.373 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:17:28.374 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:17:28.374 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:28.374 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:28.616 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:29.617 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:29.617 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:29.618 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:17:29.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:17:29.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:29.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:29.862 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:30.863 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:30.864 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:30.864 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 18 2024-09-15T21:17:30.864 INFO:tasks.workunit.client.0.smithi016.stdout:18 2024-09-15T21:17:30.864 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:30.864 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:31.114 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:32.115 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:32.116 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:32.116 INFO:tasks.workunit.client.0.smithi016.stdout:19 2024-09-15T21:17:32.116 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 19 2024-09-15T21:17:32.117 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:32.117 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:32.360 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:33.362 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:33.362 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:33.362 INFO:tasks.workunit.client.0.smithi016.stdout:20 2024-09-15T21:17:33.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 20 2024-09-15T21:17:33.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:33.363 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:17:33.611 INFO:tasks.workunit.client.0.smithi016.stdout:osd.1 up in weight 1 up_from 8 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6808/1326273600,v1:127.0.0.1:6809/1326273600] [v2:127.0.0.1:6810/1326273600,v1:127.0.0.1:6811/1326273600] exists,up 1599467e-e997-4c89-90cc-fe35cd0339c0 2024-09-15T21:17:33.614 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:17:33.614 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:17:33.614 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:17:33.614 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:172: TEST_mon_classes: run_osd td/crush-classes 2 2024-09-15T21:17:33.614 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-classes 2024-09-15T21:17:33.614 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:17:33.614 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2024-09-15T21:17:33.614 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:17:33.614 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-classes/2 2024-09-15T21:17:33.614 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false ' 2024-09-15T21:17:33.615 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:17:33.615 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:17:33.615 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:17:33.615 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-classes/2' 2024-09-15T21:17:33.615 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-classes/2/journal' 2024-09-15T21:17:33.615 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:17:33.615 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:17:33.615 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-classes' 2024-09-15T21:17:33.615 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:17:33.615 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:17:33.615 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:17:33.616 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:17:33.616 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:17:33.616 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:17:33.617 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:17:33.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:17:33.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:17:33.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:17:33.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:17:33.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-classes/$name.log' 2024-09-15T21:17:33.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-classes/$name.pid' 2024-09-15T21:17:33.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:17:33.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:17:33.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:17:33.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:17:33.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:17:33.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:17:33.619 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-classes/2 2024-09-15T21:17:33.619 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:17:33.619 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0e673255-6619-41ca-b5d8-e85a1860d7f2 2024-09-15T21:17:33.619 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 0e673255-6619-41ca-b5d8-e85a1860d7f2' 2024-09-15T21:17:33.619 INFO:tasks.workunit.client.0.smithi016.stdout:add osd2 0e673255-6619-41ca-b5d8-e85a1860d7f2 2024-09-15T21:17:33.619 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:17:33.641 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDtTudmeNA+JhAAE+8r1+5ljgbvhRSsdl7qEQ== 2024-09-15T21:17:33.641 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDtTudmeNA+JhAAE+8r1+5ljgbvhRSsdl7qEQ=="}' 2024-09-15T21:17:33.641 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0e673255-6619-41ca-b5d8-e85a1860d7f2 -i td/crush-classes/2/new.json 2024-09-15T21:17:33.942 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:17:33.958 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-classes/2/new.json 2024-09-15T21:17:33.959 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/2 --osd-journal=td/crush-classes/2/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDtTudmeNA+JhAAE+8r1+5ljgbvhRSsdl7qEQ== --osd-uuid 0e673255-6619-41ca-b5d8-e85a1860d7f2 2024-09-15T21:17:33.995 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:33.990+0000 7fb74b4c37c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:17:34.055 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:34.050+0000 7fb74b4c37c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:17:34.057 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:34.054+0000 7fb74b4c37c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:17:34.057 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:34.054+0000 7fb74b4c37c0 -1 bdev(0x557ad5235c00 td/crush-classes/2/block) open stat got: (1) Operation not permitted 2024-09-15T21:17:34.057 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:34.054+0000 7fb74b4c37c0 -1 bluestore(td/crush-classes/2) _read_fsid unparsable uuid 2024-09-15T21:17:39.685 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-classes/2/keyring 2024-09-15T21:17:39.685 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:17:39.687 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd2 key to auth repository 2024-09-15T21:17:39.687 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2024-09-15T21:17:39.687 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-classes/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:17:39.944 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2024-09-15T21:17:39.945 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.2 2024-09-15T21:17:39.945 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:17:39.945 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/2 --osd-journal=td/crush-classes/2/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:17:39.945 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:17:39.946 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:17:39.979 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:39.974+0000 7f46977c87c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:17:39.981 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:39.978+0000 7f46977c87c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:17:39.983 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:39.978+0000 7f46977c87c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:17:40.176 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2024-09-15T21:17:40.176 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:17:40.176 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2024-09-15T21:17:40.176 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:17:40.176 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:17:40.176 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:40.176 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:17:40.176 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:17:40.177 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:40.177 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:40.409 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:41.254 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:41.250+0000 7f46977c87c0 -1 Falling back to public interface 2024-09-15T21:17:41.411 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:41.411 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:41.412 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:17:41.412 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:17:41.412 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:41.412 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:41.653 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:42.655 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:42.655 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:42.655 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:17:42.656 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:17:42.656 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:42.656 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:42.897 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:43.899 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:43.899 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:43.899 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:17:43.900 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:17:43.900 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:43.900 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:43.909 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:17:43.906+0000 7f46977c87c0 -1 osd.2 0 log_to_monitors true 2024-09-15T21:17:44.450 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:45.451 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:45.452 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:45.452 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:17:45.453 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:17:45.453 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:45.453 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:45.696 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:46.698 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:46.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:46.699 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:17:46.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:17:46.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:46.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:46.944 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:47.946 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:47.946 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:47.946 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:17:47.947 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:17:47.947 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:47.947 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:48.194 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:49.195 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:49.195 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:49.195 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:17:49.196 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:17:49.196 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:49.196 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:49.440 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:50.442 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:50.442 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:50.442 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:17:50.443 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:17:50.443 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:50.443 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:50.684 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:51.686 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:51.686 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:51.686 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:17:51.687 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:17:51.687 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:51.687 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:51.927 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:52.928 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:52.928 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:52.929 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:17:52.929 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:17:52.929 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:52.929 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:53.175 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:54.176 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:54.176 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:54.177 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:17:54.177 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:17:54.177 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:54.177 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:54.424 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:55.425 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:55.426 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:55.426 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:17:55.426 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:17:55.426 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:55.426 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:55.673 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:56.674 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:56.675 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:56.675 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:17:56.675 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:17:56.675 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:56.675 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:56.927 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:57.928 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:57.929 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:57.929 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:17:57.929 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:17:57.929 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:57.930 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:58.172 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:17:59.176 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:17:59.176 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:17:59.176 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:17:59.177 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:17:59.177 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:17:59.177 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:17:59.418 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:00.420 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:00.420 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:00.421 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:18:00.421 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:18:00.421 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:00.421 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:18:00.664 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:01.666 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:01.666 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:01.667 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:18:01.667 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:18:01.667 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:01.667 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:18:01.911 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:02.913 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:02.913 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:02.913 INFO:tasks.workunit.client.0.smithi016.stdout:18 2024-09-15T21:18:02.914 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 18 2024-09-15T21:18:02.914 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:02.914 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:18:03.159 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:04.161 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:04.161 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:04.161 INFO:tasks.workunit.client.0.smithi016.stdout:19 2024-09-15T21:18:04.161 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 19 2024-09-15T21:18:04.162 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:04.162 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:18:04.405 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:05.407 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:05.407 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:05.407 INFO:tasks.workunit.client.0.smithi016.stdout:20 2024-09-15T21:18:05.408 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 20 2024-09-15T21:18:05.408 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:05.409 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:18:05.650 INFO:tasks.workunit.client.0.smithi016.stdout:osd.2 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6816/2698923629,v1:127.0.0.1:6817/2698923629] [v2:127.0.0.1:6818/2698923629,v1:127.0.0.1:6819/2698923629] exists,up 0e673255-6619-41ca-b5d8-e85a1860d7f2 2024-09-15T21:18:05.651 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:18:05.651 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:18:05.651 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:18:05.651 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:173: TEST_mon_classes: create_rbd_pool 2024-09-15T21:18:05.651 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2024-09-15T21:18:05.877 INFO:tasks.workunit.client.0.smithi016.stderr:pool 'rbd' does not exist 2024-09-15T21:18:05.893 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2024-09-15T21:18:05.893 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2024-09-15T21:18:06.283 INFO:tasks.workunit.client.0.smithi016.stderr:pool 'rbd' already exists 2024-09-15T21:18:06.302 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2024-09-15T21:18:07.305 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2024-09-15T21:18:09.262 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:175: TEST_mon_classes: get_osds_up rbd SOMETHING 2024-09-15T21:18:09.263 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:51: get_osds_up: local poolname=rbd 2024-09-15T21:18:09.263 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:52: get_osds_up: local objectname=SOMETHING 2024-09-15T21:18:09.263 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: ceph --format xml osd map rbd SOMETHING 2024-09-15T21:18:09.264 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: xmlstarlet sel -t -m //up/osd -v . -o ' ' 2024-09-15T21:18:09.515 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: local 'osds=1 2 0 ' 2024-09-15T21:18:09.515 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:57: get_osds_up: echo 1 2 0 2024-09-15T21:18:09.515 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:175: TEST_mon_classes: test '1 2 0' == '1 2 0' 2024-09-15T21:18:09.515 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:176: TEST_mon_classes: add_something td/crush-classes SOMETHING 2024-09-15T21:18:09.515 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:42: add_something: local dir=td/crush-classes 2024-09-15T21:18:09.516 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:43: add_something: local obj=SOMETHING 2024-09-15T21:18:09.516 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:45: add_something: local payload=ABCDEF 2024-09-15T21:18:09.516 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:46: add_something: echo ABCDEF 2024-09-15T21:18:09.516 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:47: add_something: rados --pool rbd put SOMETHING td/crush-classes/ORIGINAL 2024-09-15T21:18:09.668 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:179: TEST_mon_classes: ceph osd crush class create CLASS 2024-09-15T21:18:10.099 INFO:tasks.workunit.client.0.smithi016.stderr:class 'CLASS' already exists 2024-09-15T21:18:10.118 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:180: TEST_mon_classes: ceph osd crush class create CLASS 2024-09-15T21:18:10.386 INFO:tasks.workunit.client.0.smithi016.stderr:class 'CLASS' already exists 2024-09-15T21:18:10.402 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:181: TEST_mon_classes: ceph osd crush class ls 2024-09-15T21:18:10.403 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:181: TEST_mon_classes: grep CLASS 2024-09-15T21:18:10.665 INFO:tasks.workunit.client.0.smithi016.stdout: "CLASS" 2024-09-15T21:18:10.665 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:182: TEST_mon_classes: ceph osd crush class rename CLASS TEMP 2024-09-15T21:18:11.044 INFO:tasks.workunit.client.0.smithi016.stderr:already renamed to 'TEMP' 2024-09-15T21:18:11.063 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:183: TEST_mon_classes: ceph osd crush class ls 2024-09-15T21:18:11.063 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:183: TEST_mon_classes: grep TEMP 2024-09-15T21:18:11.343 INFO:tasks.workunit.client.0.smithi016.stdout: "TEMP" 2024-09-15T21:18:11.343 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:184: TEST_mon_classes: ceph osd crush class rename TEMP CLASS 2024-09-15T21:18:11.730 INFO:tasks.workunit.client.0.smithi016.stderr:already renamed to 'CLASS' 2024-09-15T21:18:11.752 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:185: TEST_mon_classes: ceph osd crush class ls 2024-09-15T21:18:11.752 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:185: TEST_mon_classes: grep CLASS 2024-09-15T21:18:12.025 INFO:tasks.workunit.client.0.smithi016.stdout: "CLASS" 2024-09-15T21:18:12.026 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:186: TEST_mon_classes: ceph osd erasure-code-profile set myprofile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd crush-device-class=CLASS 2024-09-15T21:18:12.436 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:187: TEST_mon_classes: expect_failure td/crush-classes EBUSY ceph osd crush class rm CLASS 2024-09-15T21:18:12.436 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1970: expect_failure: local dir=td/crush-classes 2024-09-15T21:18:12.436 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1971: expect_failure: shift 2024-09-15T21:18:12.436 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1972: expect_failure: local expected=EBUSY 2024-09-15T21:18:12.436 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1973: expect_failure: shift 2024-09-15T21:18:12.436 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1974: expect_failure: local success 2024-09-15T21:18:12.437 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1976: expect_failure: ceph osd crush class rm CLASS 2024-09-15T21:18:12.632 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1979: expect_failure: success=false 2024-09-15T21:18:12.632 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1982: expect_failure: false 2024-09-15T21:18:12.632 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1982: expect_failure: grep --quiet EBUSY td/crush-classes/out 2024-09-15T21:18:12.633 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1986: expect_failure: return 0 2024-09-15T21:18:12.633 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:188: TEST_mon_classes: ceph osd erasure-code-profile rm myprofile 2024-09-15T21:18:13.120 INFO:tasks.workunit.client.0.smithi016.stderr:erasure-code-profile myprofile does not exist 2024-09-15T21:18:13.137 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:189: TEST_mon_classes: ceph osd crush class rm CLASS 2024-09-15T21:18:13.540 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:190: TEST_mon_classes: ceph osd crush class rm CLASS 2024-09-15T21:18:13.830 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:193: TEST_mon_classes: ceph osd crush set-device-class aaa osd.0 2024-09-15T21:18:14.466 INFO:tasks.workunit.client.0.smithi016.stderr:osd.0 already set to class aaa. set-device-class item id 0 name 'osd.0' device_class 'aaa': no change. set osd(s) to class 'aaa' 2024-09-15T21:18:14.494 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:194: TEST_mon_classes: ceph osd tree 2024-09-15T21:18:14.494 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:194: TEST_mon_classes: grep -q aaa 2024-09-15T21:18:14.740 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:195: TEST_mon_classes: ceph osd crush dump 2024-09-15T21:18:14.740 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:195: TEST_mon_classes: grep -q '~aaa' 2024-09-15T21:18:14.982 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:196: TEST_mon_classes: ceph osd crush tree --show-shadow 2024-09-15T21:18:14.982 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:196: TEST_mon_classes: grep -q '~aaa' 2024-09-15T21:18:15.233 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:197: TEST_mon_classes: ceph osd crush set-device-class bbb osd.1 2024-09-15T21:18:15.787 INFO:tasks.workunit.client.0.smithi016.stderr:osd.1 already set to class bbb. set-device-class item id 1 name 'osd.1' device_class 'bbb': no change. set osd(s) to class 'bbb' 2024-09-15T21:18:15.815 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:198: TEST_mon_classes: ceph osd tree 2024-09-15T21:18:15.815 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:198: TEST_mon_classes: grep -q bbb 2024-09-15T21:18:16.254 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:199: TEST_mon_classes: ceph osd crush dump 2024-09-15T21:18:16.254 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:199: TEST_mon_classes: grep -q '~bbb' 2024-09-15T21:18:16.499 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:200: TEST_mon_classes: ceph osd crush tree --show-shadow 2024-09-15T21:18:16.499 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:200: TEST_mon_classes: grep -q '~bbb' 2024-09-15T21:18:16.751 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:201: TEST_mon_classes: ceph osd crush set-device-class ccc osd.2 2024-09-15T21:18:17.259 INFO:tasks.workunit.client.0.smithi016.stderr:osd.2 already set to class ccc. set-device-class item id 2 name 'osd.2' device_class 'ccc': no change. set osd(s) to class 'ccc' 2024-09-15T21:18:17.281 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:202: TEST_mon_classes: ceph osd tree 2024-09-15T21:18:17.281 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:202: TEST_mon_classes: grep -q ccc 2024-09-15T21:18:17.614 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:203: TEST_mon_classes: ceph osd crush dump 2024-09-15T21:18:17.615 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:203: TEST_mon_classes: grep -q '~ccc' 2024-09-15T21:18:17.869 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:204: TEST_mon_classes: grep -q '~ccc' 2024-09-15T21:18:17.870 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:204: TEST_mon_classes: ceph osd crush tree --show-shadow 2024-09-15T21:18:18.123 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:205: TEST_mon_classes: ceph osd crush rm-device-class 0 2024-09-15T21:18:18.639 INFO:tasks.workunit.client.0.smithi016.stderr:osd.0 belongs to no class, done removing class of osd(s): 2024-09-15T21:18:18.663 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:206: TEST_mon_classes: grep -q aaa 2024-09-15T21:18:18.663 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:206: TEST_mon_classes: ceph osd tree 2024-09-15T21:18:18.965 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:207: TEST_mon_classes: ceph osd crush class ls 2024-09-15T21:18:18.965 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:207: TEST_mon_classes: grep -q aaa 2024-09-15T21:18:19.214 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:208: TEST_mon_classes: ceph osd crush rm-device-class 1 2024-09-15T21:18:19.844 INFO:tasks.workunit.client.0.smithi016.stderr:osd.1 belongs to no class, done removing class of osd(s): 2024-09-15T21:18:19.865 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:209: TEST_mon_classes: ceph osd tree 2024-09-15T21:18:19.865 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:209: TEST_mon_classes: grep -q bbb 2024-09-15T21:18:20.189 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:210: TEST_mon_classes: ceph osd crush class ls 2024-09-15T21:18:20.189 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:210: TEST_mon_classes: grep -q bbb 2024-09-15T21:18:20.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:211: TEST_mon_classes: ceph osd crush rm-device-class 2 2024-09-15T21:18:20.964 INFO:tasks.workunit.client.0.smithi016.stderr:osd.2 belongs to no class, done removing class of osd(s): 2024-09-15T21:18:20.986 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:212: TEST_mon_classes: ceph osd tree 2024-09-15T21:18:20.986 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:212: TEST_mon_classes: grep -q ccc 2024-09-15T21:18:21.294 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:213: TEST_mon_classes: ceph osd crush class ls 2024-09-15T21:18:21.295 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:213: TEST_mon_classes: grep -q ccc 2024-09-15T21:18:21.557 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:214: TEST_mon_classes: ceph osd crush set-device-class asdf all 2024-09-15T21:18:22.127 INFO:tasks.workunit.client.0.smithi016.stderr:osd.0 already set to class asdf. osd.1 already set to class asdf. osd.2 already set to class asdf. set osd(s) to class 'asdf' 2024-09-15T21:18:22.151 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:215: TEST_mon_classes: ceph osd tree 2024-09-15T21:18:22.151 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:215: TEST_mon_classes: grep -q asdf 2024-09-15T21:18:22.474 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:216: TEST_mon_classes: ceph osd crush dump 2024-09-15T21:18:22.474 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:216: TEST_mon_classes: grep -q '~asdf' 2024-09-15T21:18:22.730 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:217: TEST_mon_classes: ceph osd crush tree --show-shadow 2024-09-15T21:18:22.731 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:217: TEST_mon_classes: grep -q '~asdf' 2024-09-15T21:18:22.979 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:218: TEST_mon_classes: ceph osd crush rule create-replicated asdf-rule default host asdf 2024-09-15T21:18:23.373 INFO:tasks.workunit.client.0.smithi016.stderr:rule asdf-rule already exists 2024-09-15T21:18:23.394 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:219: TEST_mon_classes: ceph osd crush rm-device-class all 2024-09-15T21:18:23.930 INFO:tasks.workunit.client.0.smithi016.stderr:osd.0 belongs to no class, osd.1 belongs to no class, osd.2 belongs to no class, done removing class of osd(s): 2024-09-15T21:18:23.954 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:220: TEST_mon_classes: ceph osd tree 2024-09-15T21:18:23.954 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:220: TEST_mon_classes: grep -q asdf 2024-09-15T21:18:24.245 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:221: TEST_mon_classes: ceph osd crush class ls 2024-09-15T21:18:24.246 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:221: TEST_mon_classes: grep -q asdf 2024-09-15T21:18:24.505 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:223: TEST_mon_classes: ceph osd crush set-device-class abc osd.2 2024-09-15T21:18:25.042 INFO:tasks.workunit.client.0.smithi016.stderr:osd.2 already set to class abc. set-device-class item id 2 name 'osd.2' device_class 'abc': no change. set osd(s) to class 'abc' 2024-09-15T21:18:25.066 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:224: TEST_mon_classes: ceph osd crush move osd.2 root=foo rack=foo-rack host=foo-host 2024-09-15T21:18:25.466 INFO:tasks.workunit.client.0.smithi016.stderr:no need to move item id 2 name 'osd.2' to location {host=foo-host,rack=foo-rack,root=foo} in crush map 2024-09-15T21:18:25.484 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:225: TEST_mon_classes: ceph osd tree 2024-09-15T21:18:25.484 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:225: TEST_mon_classes: awk '$1 == 2 && $2 == "abc" {print $0}' 2024-09-15T21:18:25.950 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:225: TEST_mon_classes: out=' 2 abc 0.09769 osd.2 up 1.00000 1.00000' 2024-09-15T21:18:25.951 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:226: TEST_mon_classes: '[' ' 2 abc 0.09769 osd.2 up 1.00000 1.00000' == '' ']' 2024-09-15T21:18:25.951 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:231: TEST_mon_classes: ceph osd crush dump 2024-09-15T21:18:25.951 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:231: TEST_mon_classes: grep -q foo~abc 2024-09-15T21:18:26.308 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:232: TEST_mon_classes: grep -q foo~abc 2024-09-15T21:18:26.308 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:232: TEST_mon_classes: ceph osd crush tree --show-shadow 2024-09-15T21:18:26.562 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:233: TEST_mon_classes: ceph osd crush dump 2024-09-15T21:18:26.562 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:233: TEST_mon_classes: grep -q foo-rack~abc 2024-09-15T21:18:26.898 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:234: TEST_mon_classes: ceph osd crush tree --show-shadow 2024-09-15T21:18:26.898 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:234: TEST_mon_classes: grep -q foo-rack~abc 2024-09-15T21:18:27.228 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:235: TEST_mon_classes: ceph osd crush dump 2024-09-15T21:18:27.228 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:235: TEST_mon_classes: grep -q foo-host~abc 2024-09-15T21:18:27.463 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:236: TEST_mon_classes: ceph osd crush tree --show-shadow 2024-09-15T21:18:27.464 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:236: TEST_mon_classes: grep -q foo-host~abc 2024-09-15T21:18:27.714 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:237: TEST_mon_classes: ceph osd crush rm-device-class osd.2 2024-09-15T21:18:28.264 INFO:tasks.workunit.client.0.smithi016.stderr:osd.2 belongs to no class, done removing class of osd(s): 2024-09-15T21:18:28.287 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:239: TEST_mon_classes: ceph osd crush set-device-class abc osd.2 2024-09-15T21:18:28.799 INFO:tasks.workunit.client.0.smithi016.stderr:osd.2 already set to class abc. set-device-class item id 2 name 'osd.2' device_class 'abc': no change. set osd(s) to class 'abc' 2024-09-15T21:18:28.819 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:241: TEST_mon_classes: ceph osd crush rule create-replicated foo-rule foo host abc 2024-09-15T21:18:29.218 INFO:tasks.workunit.client.0.smithi016.stderr:rule foo-rule already exists 2024-09-15T21:18:29.235 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:244: TEST_mon_classes: ceph osd crush set-device-class hdd osd.0 2024-09-15T21:18:29.763 INFO:tasks.workunit.client.0.smithi016.stderr:osd.0 already set to class hdd. set-device-class item id 0 name 'osd.0' device_class 'hdd': no change. set osd(s) to class 'hdd' 2024-09-15T21:18:29.786 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:245: TEST_mon_classes: expect_failure td/crush-classes EBUSY ceph osd crush set-device-class nvme osd.0 2024-09-15T21:18:29.786 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1970: expect_failure: local dir=td/crush-classes 2024-09-15T21:18:29.786 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1971: expect_failure: shift 2024-09-15T21:18:29.786 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1972: expect_failure: local expected=EBUSY 2024-09-15T21:18:29.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1973: expect_failure: shift 2024-09-15T21:18:29.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1974: expect_failure: local success 2024-09-15T21:18:29.787 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1976: expect_failure: ceph osd crush set-device-class nvme osd.0 2024-09-15T21:18:29.986 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1979: expect_failure: success=false 2024-09-15T21:18:29.986 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1982: expect_failure: false 2024-09-15T21:18:29.986 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1982: expect_failure: grep --quiet EBUSY td/crush-classes/out 2024-09-15T21:18:29.987 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1986: expect_failure: return 0 2024-09-15T21:18:29.987 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:248: TEST_mon_classes: ceph osd crush rm-device-class all 2024-09-15T21:18:30.516 INFO:tasks.workunit.client.0.smithi016.stderr:osd.0 belongs to no class, osd.1 belongs to no class, osd.2 belongs to no class, done removing class of osd(s): 2024-09-15T21:18:30.538 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:249: TEST_mon_classes: ceph osd crush set-device-class class_1 all 2024-09-15T21:18:31.219 INFO:tasks.workunit.client.0.smithi016.stderr:osd.0 already set to class class_1. osd.1 already set to class class_1. osd.2 already set to class class_1. set osd(s) to class 'class_1' 2024-09-15T21:18:31.245 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:250: TEST_mon_classes: ceph osd crush class ls 2024-09-15T21:18:31.245 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:250: TEST_mon_classes: grep class_1 2024-09-15T21:18:31.556 INFO:tasks.workunit.client.0.smithi016.stdout: "class_1" 2024-09-15T21:18:31.557 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:251: TEST_mon_classes: ceph osd crush tree --show-shadow 2024-09-15T21:18:31.557 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:251: TEST_mon_classes: grep class_1 2024-09-15T21:18:31.822 INFO:tasks.workunit.client.0.smithi016.stdout:-20 class_1 0.19537 root default~class_1 2024-09-15T21:18:31.823 INFO:tasks.workunit.client.0.smithi016.stdout:-19 class_1 0.19537 host smithi016~class_1 2024-09-15T21:18:31.823 INFO:tasks.workunit.client.0.smithi016.stdout: 0 class_1 0.09769 osd.0 2024-09-15T21:18:31.823 INFO:tasks.workunit.client.0.smithi016.stdout: 1 class_1 0.09769 osd.1 2024-09-15T21:18:31.823 INFO:tasks.workunit.client.0.smithi016.stdout:-18 class_1 0.09769 root foo~class_1 2024-09-15T21:18:31.823 INFO:tasks.workunit.client.0.smithi016.stdout:-17 class_1 0.09769 rack foo-rack~class_1 2024-09-15T21:18:31.823 INFO:tasks.workunit.client.0.smithi016.stdout:-16 class_1 0.09769 host foo-host~class_1 2024-09-15T21:18:31.823 INFO:tasks.workunit.client.0.smithi016.stdout: 2 class_1 0.09769 osd.2 2024-09-15T21:18:31.823 INFO:tasks.workunit.client.0.smithi016.stdout: 2 class_1 0.09769 osd.2 2024-09-15T21:18:31.823 INFO:tasks.workunit.client.0.smithi016.stdout: 0 class_1 0.09769 osd.0 2024-09-15T21:18:31.823 INFO:tasks.workunit.client.0.smithi016.stdout: 1 class_1 0.09769 osd.1 2024-09-15T21:18:31.823 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:252: TEST_mon_classes: ceph osd crush rule create-replicated class_1_rule default host class_1 2024-09-15T21:18:32.206 INFO:tasks.workunit.client.0.smithi016.stderr:rule class_1_rule already exists 2024-09-15T21:18:32.226 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:253: TEST_mon_classes: ceph osd crush class rename class_1 class_2 2024-09-15T21:18:32.678 INFO:tasks.workunit.client.0.smithi016.stderr:already renamed to 'class_2' 2024-09-15T21:18:32.696 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:254: TEST_mon_classes: ceph osd crush class rename class_1 class_2 2024-09-15T21:18:32.938 INFO:tasks.workunit.client.0.smithi016.stderr:already renamed to 'class_2' 2024-09-15T21:18:32.954 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:255: TEST_mon_classes: ceph osd crush class ls 2024-09-15T21:18:32.954 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:255: TEST_mon_classes: grep class_1 2024-09-15T21:18:33.205 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:256: TEST_mon_classes: grep class_1 2024-09-15T21:18:33.205 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:256: TEST_mon_classes: ceph osd crush tree --show-shadow 2024-09-15T21:18:33.460 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:257: TEST_mon_classes: ceph osd crush class ls 2024-09-15T21:18:33.460 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:257: TEST_mon_classes: grep class_2 2024-09-15T21:18:33.713 INFO:tasks.workunit.client.0.smithi016.stdout: "class_2" 2024-09-15T21:18:33.713 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:258: TEST_mon_classes: ceph osd crush tree --show-shadow 2024-09-15T21:18:33.714 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:258: TEST_mon_classes: grep class_2 2024-09-15T21:18:33.967 INFO:tasks.workunit.client.0.smithi016.stdout:-20 class_2 0.19537 root default~class_2 2024-09-15T21:18:33.967 INFO:tasks.workunit.client.0.smithi016.stdout:-19 class_2 0.19537 host smithi016~class_2 2024-09-15T21:18:33.967 INFO:tasks.workunit.client.0.smithi016.stdout: 0 class_2 0.09769 osd.0 2024-09-15T21:18:33.967 INFO:tasks.workunit.client.0.smithi016.stdout: 1 class_2 0.09769 osd.1 2024-09-15T21:18:33.967 INFO:tasks.workunit.client.0.smithi016.stdout:-18 class_2 0.09769 root foo~class_2 2024-09-15T21:18:33.967 INFO:tasks.workunit.client.0.smithi016.stdout:-17 class_2 0.09769 rack foo-rack~class_2 2024-09-15T21:18:33.967 INFO:tasks.workunit.client.0.smithi016.stdout:-16 class_2 0.09769 host foo-host~class_2 2024-09-15T21:18:33.967 INFO:tasks.workunit.client.0.smithi016.stdout: 2 class_2 0.09769 osd.2 2024-09-15T21:18:33.967 INFO:tasks.workunit.client.0.smithi016.stdout: 2 class_2 0.09769 osd.2 2024-09-15T21:18:33.967 INFO:tasks.workunit.client.0.smithi016.stdout: 0 class_2 0.09769 osd.0 2024-09-15T21:18:33.967 INFO:tasks.workunit.client.0.smithi016.stdout: 1 class_2 0.09769 osd.1 2024-09-15T21:18:33.968 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:37: run: teardown td/crush-classes 2024-09-15T21:18:33.969 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-classes 2024-09-15T21:18:33.969 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:18:33.969 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-classes KILL 2024-09-15T21:18:33.969 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:18:33.969 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:18:33.969 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:18:33.969 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:18:33.969 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:18:34.078 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:18:34.079 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:18:34.080 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:18:34.081 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:18:34.082 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:18:34.082 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:18:34.082 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:18:34.083 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:18:34.083 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:18:34.084 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:18:34.084 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:18:34.085 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:18:34.086 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:18:34.086 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-classes 2024-09-15T21:18:34.101 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:18:34.101 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:18:34.101 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:18:34.101 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39842 2024-09-15T21:18:34.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:18:34.102 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:18:34.103 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:34: run: for func in $funcs 2024-09-15T21:18:34.103 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:35: run: setup td/crush-classes 2024-09-15T21:18:34.103 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/crush-classes 2024-09-15T21:18:34.103 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/crush-classes 2024-09-15T21:18:34.103 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-classes 2024-09-15T21:18:34.103 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:18:34.103 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-classes KILL 2024-09-15T21:18:34.104 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:18:34.104 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:18:34.104 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:18:34.104 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:18:34.104 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:18:34.105 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:18:34.106 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:18:34.106 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:18:34.107 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:18:34.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:18:34.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:18:34.109 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:18:34.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:18:34.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:18:34.110 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:18:34.110 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:18:34.111 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:18:34.112 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:18:34.113 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-classes 2024-09-15T21:18:34.114 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:18:34.114 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:18:34.114 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:18:34.114 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39842 2024-09-15T21:18:34.115 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:18:34.115 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:18:34.115 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/crush-classes 2024-09-15T21:18:34.117 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2024-09-15T21:18:34.117 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:18:34.117 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:18:34.117 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.39842 2024-09-15T21:18:34.120 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2024-09-15T21:18:34.120 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2024-09-15T21:18:34.120 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2024-09-15T21:18:34.120 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/crush-classes 1' TERM HUP INT 2024-09-15T21:18:34.120 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:36: run: TEST_reweight_vs_classes td/crush-classes 2024-09-15T21:18:34.120 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:61: TEST_reweight_vs_classes: local dir=td/crush-classes 2024-09-15T21:18:34.120 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:66: TEST_reweight_vs_classes: run_mon td/crush-classes a 2024-09-15T21:18:34.120 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/crush-classes 2024-09-15T21:18:34.120 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2024-09-15T21:18:34.121 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2024-09-15T21:18:34.121 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2024-09-15T21:18:34.121 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/crush-classes/a 2024-09-15T21:18:34.121 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/crush-classes/a --run-dir=td/crush-classes 2024-09-15T21:18:34.580 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2024-09-15T21:18:34.580 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:18:34.580 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:18:34.580 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:18:34.581 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:18:34.581 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:18:34.581 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:18:34.581 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/crush-classes/a '--log-file=td/crush-classes/$name.log' '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --mon-cluster-log-file=td/crush-classes/log --run-dir=td/crush-classes '--pid-file=td/crush-classes/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2024-09-15T21:18:34.825 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2024-09-15T21:18:34.827 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2024-09-15T21:18:34.827 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:18:34.827 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:18:34.827 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2024-09-15T21:18:34.828 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2024-09-15T21:18:34.828 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:18:34.828 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:18:34.828 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:18:34.829 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:18:34.829 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:18:34.829 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:18:34.829 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39842/ceph-mon.a.asok 2024-09-15T21:18:34.829 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:18:34.829 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39842/ceph-mon.a.asok config get fsid 2024-09-15T21:18:34.948 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2024-09-15T21:18:34.949 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:18:34.949 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:18:34.949 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2024-09-15T21:18:34.949 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2024-09-15T21:18:34.950 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:18:34.950 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:18:34.950 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:18:34.950 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:18:34.950 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:18:34.950 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:18:34.950 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39842/ceph-mon.a.asok 2024-09-15T21:18:34.951 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:18:34.951 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39842/ceph-mon.a.asok config get mon_host 2024-09-15T21:18:35.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:67: TEST_reweight_vs_classes: run_osd td/crush-classes 0 2024-09-15T21:18:35.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-classes 2024-09-15T21:18:35.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:18:35.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2024-09-15T21:18:35.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:18:35.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-classes/0 2024-09-15T21:18:35.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false ' 2024-09-15T21:18:35.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:18:35.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:18:35.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:18:35.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-classes/0' 2024-09-15T21:18:35.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-classes/0/journal' 2024-09-15T21:18:35.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:18:35.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:18:35.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-classes' 2024-09-15T21:18:35.070 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:18:35.070 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:18:35.070 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:18:35.070 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:18:35.070 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:18:35.070 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:18:35.070 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:18:35.072 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:18:35.072 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:18:35.072 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:18:35.072 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:18:35.072 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-classes/$name.log' 2024-09-15T21:18:35.072 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-classes/$name.pid' 2024-09-15T21:18:35.072 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:18:35.072 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:18:35.072 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:18:35.072 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:18:35.072 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:18:35.072 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:18:35.073 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-classes/0 2024-09-15T21:18:35.073 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:18:35.073 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=88ad4303-51dc-4b92-a9ab-76c9aeffff77 2024-09-15T21:18:35.073 INFO:tasks.workunit.client.0.smithi016.stdout:add osd0 88ad4303-51dc-4b92-a9ab-76c9aeffff77 2024-09-15T21:18:35.074 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 88ad4303-51dc-4b92-a9ab-76c9aeffff77' 2024-09-15T21:18:35.074 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:18:35.096 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQArT+dmSbjABRAAuBz5zusMECmEzGQiLsIKWg== 2024-09-15T21:18:35.096 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQArT+dmSbjABRAAuBz5zusMECmEzGQiLsIKWg=="}' 2024-09-15T21:18:35.096 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 88ad4303-51dc-4b92-a9ab-76c9aeffff77 -i td/crush-classes/0/new.json 2024-09-15T21:18:36.044 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:18:36.062 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-classes/0/new.json 2024-09-15T21:18:36.063 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/0 --osd-journal=td/crush-classes/0/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQArT+dmSbjABRAAuBz5zusMECmEzGQiLsIKWg== --osd-uuid 88ad4303-51dc-4b92-a9ab-76c9aeffff77 2024-09-15T21:18:36.102 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:18:36.101+0000 7fa8d89397c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:18:36.121 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:18:36.121+0000 7fa8d89397c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:18:36.123 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:18:36.121+0000 7fa8d89397c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:18:36.123 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:18:36.121+0000 7fa8d89397c0 -1 bdev(0x562ed3369c00 td/crush-classes/0/block) open stat got: (1) Operation not permitted 2024-09-15T21:18:36.123 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:18:36.121+0000 7fa8d89397c0 -1 bluestore(td/crush-classes/0) _read_fsid unparsable uuid 2024-09-15T21:18:43.326 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-classes/0/keyring 2024-09-15T21:18:43.326 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:18:43.327 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd0 key to auth repository 2024-09-15T21:18:43.327 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2024-09-15T21:18:43.328 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-classes/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:18:43.584 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2024-09-15T21:18:43.584 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.0 2024-09-15T21:18:43.585 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/0 --osd-journal=td/crush-classes/0/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:18:43.585 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:18:43.585 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:18:43.585 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:18:43.611 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:18:43.609+0000 7f22f662a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:18:43.622 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:18:43.621+0000 7f22f662a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:18:43.624 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:18:43.625+0000 7f22f662a7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:18:43.829 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2024-09-15T21:18:43.829 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:18:43.829 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2024-09-15T21:18:43.829 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:18:43.829 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:18:43.829 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:43.829 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:18:43.830 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:18:43.830 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:43.830 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:18:44.076 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:44.907 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:18:44.905+0000 7f22f662a7c0 -1 Falling back to public interface 2024-09-15T21:18:45.078 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:45.078 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:45.078 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:18:45.078 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:18:45.079 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:45.079 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:18:45.323 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:46.324 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:46.325 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:46.325 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:18:46.325 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:18:46.325 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:46.326 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:18:46.569 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:47.571 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:47.571 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:47.571 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:18:47.572 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:18:47.572 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:47.572 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:18:47.767 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:18:47.765+0000 7f22f662a7c0 -1 osd.0 0 log_to_monitors true 2024-09-15T21:18:47.817 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:48.819 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:48.819 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:48.819 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:18:48.820 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:18:48.821 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:48.821 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:18:49.064 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:50.065 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:50.066 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:50.066 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:18:50.066 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:18:50.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:50.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:18:50.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:51.315 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:51.316 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:51.316 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:18:51.316 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:18:51.316 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:51.316 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:18:51.566 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:52.567 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:52.567 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:52.568 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:18:52.568 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:18:52.568 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:52.568 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:18:52.812 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:53.813 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:53.814 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:53.814 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:18:53.814 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:18:53.815 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:53.815 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:18:54.060 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:55.061 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:55.061 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:55.062 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:18:55.062 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:18:55.062 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:55.062 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:18:55.306 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:56.308 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:56.308 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:56.309 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:18:56.309 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:18:56.309 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:56.309 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:18:56.553 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:57.557 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:57.557 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:57.557 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:18:57.558 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:18:57.558 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:57.558 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:18:57.798 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:18:58.801 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:18:58.801 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:18:58.801 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:18:58.802 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:18:58.802 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:18:58.802 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:18:59.049 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:00.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:00.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:00.052 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:19:00.052 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:19:00.052 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:00.053 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:19:00.296 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:01.297 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:01.297 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:01.298 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:19:01.298 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:19:01.298 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:01.298 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:19:01.543 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:02.545 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:02.545 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:02.545 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:19:02.546 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:19:02.546 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:02.546 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:19:02.794 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:03.795 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:03.796 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:03.796 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:19:03.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:19:03.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:03.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:19:04.037 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:05.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:05.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:05.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:19:05.039 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:19:05.040 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:05.040 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:19:05.286 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:06.288 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:06.288 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:06.288 INFO:tasks.workunit.client.0.smithi016.stdout:18 2024-09-15T21:19:06.289 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 18 2024-09-15T21:19:06.289 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:06.289 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:19:06.531 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:07.533 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:07.533 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:07.533 INFO:tasks.workunit.client.0.smithi016.stdout:19 2024-09-15T21:19:07.534 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 19 2024-09-15T21:19:07.534 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:07.534 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:19:07.776 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:08.776 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:08.776 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:08.776 INFO:tasks.workunit.client.0.smithi016.stdout:20 2024-09-15T21:19:08.777 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 20 2024-09-15T21:19:08.777 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:08.777 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:19:09.018 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:10.020 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:10.021 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:10.024 INFO:tasks.workunit.client.0.smithi016.stdout:21 2024-09-15T21:19:10.024 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 21 2024-09-15T21:19:10.025 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:10.025 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:19:10.268 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:11.270 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:11.270 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:11.271 INFO:tasks.workunit.client.0.smithi016.stdout:22 2024-09-15T21:19:11.271 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 22 2024-09-15T21:19:11.271 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:11.271 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:19:11.513 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:12.515 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:12.515 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:12.516 INFO:tasks.workunit.client.0.smithi016.stdout:23 2024-09-15T21:19:12.517 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 23 2024-09-15T21:19:12.517 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:12.517 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:19:12.762 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:13.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:13.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:13.764 INFO:tasks.workunit.client.0.smithi016.stdout:24 2024-09-15T21:19:13.765 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 24 2024-09-15T21:19:13.765 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:13.765 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:19:14.068 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:15.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:15.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:15.070 INFO:tasks.workunit.client.0.smithi016.stdout:25 2024-09-15T21:19:15.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 25 2024-09-15T21:19:15.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:15.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:19:15.314 INFO:tasks.workunit.client.0.smithi016.stdout:osd.0 up in weight 1 up_from 4 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3835936840,v1:127.0.0.1:6801/3835936840] [v2:127.0.0.1:6802/3835936840,v1:127.0.0.1:6803/3835936840] exists,up 88ad4303-51dc-4b92-a9ab-76c9aeffff77 2024-09-15T21:19:15.316 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:19:15.316 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:19:15.316 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:19:15.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:68: TEST_reweight_vs_classes: run_osd td/crush-classes 1 2024-09-15T21:19:15.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-classes 2024-09-15T21:19:15.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:19:15.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2024-09-15T21:19:15.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:19:15.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-classes/1 2024-09-15T21:19:15.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false ' 2024-09-15T21:19:15.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:19:15.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:19:15.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:19:15.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-classes/1' 2024-09-15T21:19:15.317 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-classes/1/journal' 2024-09-15T21:19:15.318 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:19:15.318 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:19:15.318 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-classes' 2024-09-15T21:19:15.318 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:19:15.318 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:19:15.318 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:19:15.318 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:19:15.318 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:19:15.318 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:19:15.320 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:19:15.320 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:19:15.320 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:19:15.320 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:19:15.320 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:19:15.320 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-classes/$name.log' 2024-09-15T21:19:15.320 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-classes/$name.pid' 2024-09-15T21:19:15.320 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:19:15.320 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:19:15.320 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:19:15.320 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:19:15.321 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:19:15.321 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:19:15.321 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-classes/1 2024-09-15T21:19:15.321 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:19:15.321 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d6d56c08-e908-42bd-ba59-6e1cd1ef720b 2024-09-15T21:19:15.322 INFO:tasks.workunit.client.0.smithi016.stdout:add osd1 d6d56c08-e908-42bd-ba59-6e1cd1ef720b 2024-09-15T21:19:15.322 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 d6d56c08-e908-42bd-ba59-6e1cd1ef720b' 2024-09-15T21:19:15.322 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:19:15.344 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBTT+dmtLCLFBAAHC9PzMTv1HVabmNk2c/stg== 2024-09-15T21:19:15.344 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBTT+dmtLCLFBAAHC9PzMTv1HVabmNk2c/stg=="}' 2024-09-15T21:19:15.344 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d6d56c08-e908-42bd-ba59-6e1cd1ef720b -i td/crush-classes/1/new.json 2024-09-15T21:19:15.754 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:19:15.774 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-classes/1/new.json 2024-09-15T21:19:15.775 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/1 --osd-journal=td/crush-classes/1/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBTT+dmtLCLFBAAHC9PzMTv1HVabmNk2c/stg== --osd-uuid d6d56c08-e908-42bd-ba59-6e1cd1ef720b 2024-09-15T21:19:15.811 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:15.809+0000 7fbcbab207c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:19:15.848 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:15.849+0000 7fbcbab207c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:19:15.850 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:15.849+0000 7fbcbab207c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:19:15.850 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:15.849+0000 7fbcbab207c0 -1 bdev(0x55e3fe607c00 td/crush-classes/1/block) open stat got: (1) Operation not permitted 2024-09-15T21:19:15.850 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:15.849+0000 7fbcbab207c0 -1 bluestore(td/crush-classes/1) _read_fsid unparsable uuid 2024-09-15T21:19:23.018 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-classes/1/keyring 2024-09-15T21:19:23.019 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:19:23.020 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd1 key to auth repository 2024-09-15T21:19:23.021 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2024-09-15T21:19:23.021 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-classes/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:19:23.279 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2024-09-15T21:19:23.279 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.1 2024-09-15T21:19:23.279 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/1 --osd-journal=td/crush-classes/1/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:19:23.280 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:19:23.280 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:19:23.280 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:19:23.305 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:23.305+0000 7f49a62ca7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:19:23.307 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:23.305+0000 7f49a62ca7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:19:23.308 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:23.309+0000 7f49a62ca7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:19:23.517 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2024-09-15T21:19:23.517 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:19:23.517 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2024-09-15T21:19:23.517 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:19:23.517 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:19:23.517 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:23.518 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:19:23.519 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:19:23.520 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:23.520 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:23.750 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:24.751 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:24.751 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:24.752 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:19:24.752 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:19:24.752 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:24.752 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:24.854 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:24.853+0000 7f49a62ca7c0 -1 Falling back to public interface 2024-09-15T21:19:24.993 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:25.994 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:25.995 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:25.995 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:19:25.995 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:19:25.995 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:25.995 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:26.239 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:27.241 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:27.241 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:27.241 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:19:27.242 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:19:27.242 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:27.242 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:27.484 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:27.871 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:27.869+0000 7f49a62ca7c0 -1 osd.1 0 log_to_monitors true 2024-09-15T21:19:28.486 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:28.486 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:28.486 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:19:28.487 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:19:28.487 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:28.487 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:28.843 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:29.845 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:29.845 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:29.846 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:19:29.846 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:19:29.846 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:29.846 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:30.095 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:31.097 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:31.097 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:31.097 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:19:31.098 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:19:31.098 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:31.098 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:31.341 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:32.343 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:32.343 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:32.343 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:19:32.344 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:19:32.344 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:32.344 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:32.587 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:33.589 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:33.589 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:33.590 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:19:33.590 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:19:33.590 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:33.590 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:33.831 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:34.833 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:34.833 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:34.834 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:19:34.834 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:19:34.835 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:34.835 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:35.082 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:36.084 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:36.084 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:36.084 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:19:36.085 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:19:36.085 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:36.085 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:36.328 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:37.329 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:37.329 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:37.329 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:19:37.330 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:19:37.330 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:37.330 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:37.572 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:38.573 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:38.573 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:38.574 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:19:38.574 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:19:38.574 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:38.574 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:38.816 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:39.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:39.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:39.818 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:19:39.819 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:19:39.819 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:39.819 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:40.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:41.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:41.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:41.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:19:41.069 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:19:41.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:41.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:41.313 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:42.315 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:42.315 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:42.315 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:19:42.315 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:19:42.316 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:42.316 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:42.561 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:43.563 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:43.563 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:43.563 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:19:43.564 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:19:43.564 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:43.564 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:43.813 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:44.817 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:44.817 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:44.817 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:19:44.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:19:44.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:44.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:45.058 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:46.060 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:46.060 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:46.060 INFO:tasks.workunit.client.0.smithi016.stdout:18 2024-09-15T21:19:46.061 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 18 2024-09-15T21:19:46.061 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:46.061 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:46.302 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:47.304 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:47.304 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:47.304 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 19 2024-09-15T21:19:47.305 INFO:tasks.workunit.client.0.smithi016.stdout:19 2024-09-15T21:19:47.305 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:47.305 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:47.548 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:48.549 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:48.549 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:48.550 INFO:tasks.workunit.client.0.smithi016.stdout:20 2024-09-15T21:19:48.550 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 20 2024-09-15T21:19:48.550 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:48.550 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:48.793 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:49.794 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:49.794 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:49.795 INFO:tasks.workunit.client.0.smithi016.stdout:21 2024-09-15T21:19:49.795 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 21 2024-09-15T21:19:49.795 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:49.795 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:50.040 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:51.042 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:51.042 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:51.042 INFO:tasks.workunit.client.0.smithi016.stdout:22 2024-09-15T21:19:51.043 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 22 2024-09-15T21:19:51.043 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:51.043 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:51.287 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:52.289 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:52.289 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:52.289 INFO:tasks.workunit.client.0.smithi016.stdout:23 2024-09-15T21:19:52.290 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 23 2024-09-15T21:19:52.290 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:52.290 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:52.536 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:53.537 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:53.537 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:53.538 INFO:tasks.workunit.client.0.smithi016.stdout:24 2024-09-15T21:19:53.538 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 24 2024-09-15T21:19:53.538 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:53.538 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:53.817 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:19:54.819 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:19:54.819 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:19:54.820 INFO:tasks.workunit.client.0.smithi016.stdout:25 2024-09-15T21:19:54.820 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 25 2024-09-15T21:19:54.820 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:19:54.820 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:19:55.086 INFO:tasks.workunit.client.0.smithi016.stdout:osd.1 up in weight 1 up_from 8 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6808/1382889312,v1:127.0.0.1:6809/1382889312] [v2:127.0.0.1:6810/1382889312,v1:127.0.0.1:6811/1382889312] exists,up d6d56c08-e908-42bd-ba59-6e1cd1ef720b 2024-09-15T21:19:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:19:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:19:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:19:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:69: TEST_reweight_vs_classes: run_osd td/crush-classes 2 2024-09-15T21:19:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-classes 2024-09-15T21:19:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:19:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2024-09-15T21:19:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:19:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-classes/2 2024-09-15T21:19:55.090 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false ' 2024-09-15T21:19:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:19:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:19:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:19:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-classes/2' 2024-09-15T21:19:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-classes/2/journal' 2024-09-15T21:19:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:19:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:19:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-classes' 2024-09-15T21:19:55.091 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:19:55.091 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:19:55.091 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:19:55.091 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:19:55.092 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:19:55.092 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:19:55.092 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:19:55.092 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:19:55.092 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:19:55.092 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:19:55.092 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:19:55.092 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-classes/$name.log' 2024-09-15T21:19:55.092 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-classes/$name.pid' 2024-09-15T21:19:55.092 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:19:55.093 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:19:55.093 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:19:55.093 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:19:55.093 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:19:55.093 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:19:55.093 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-classes/2 2024-09-15T21:19:55.094 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:19:55.094 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5f9145f0-71b3-4f67-9c8b-47d461069f4f 2024-09-15T21:19:55.094 INFO:tasks.workunit.client.0.smithi016.stdout:add osd2 5f9145f0-71b3-4f67-9c8b-47d461069f4f 2024-09-15T21:19:55.095 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 5f9145f0-71b3-4f67-9c8b-47d461069f4f' 2024-09-15T21:19:55.095 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:19:55.111 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB7T+dmzNK0BhAAmBmX2AdAvKdb9pL6rDMjew== 2024-09-15T21:19:55.112 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB7T+dmzNK0BhAAmBmX2AdAvKdb9pL6rDMjew=="}' 2024-09-15T21:19:55.112 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5f9145f0-71b3-4f67-9c8b-47d461069f4f -i td/crush-classes/2/new.json 2024-09-15T21:19:55.442 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:19:55.458 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-classes/2/new.json 2024-09-15T21:19:55.460 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/2 --osd-journal=td/crush-classes/2/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB7T+dmzNK0BhAAmBmX2AdAvKdb9pL6rDMjew== --osd-uuid 5f9145f0-71b3-4f67-9c8b-47d461069f4f 2024-09-15T21:19:55.497 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:55.496+0000 7f5e5c7c87c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:19:55.499 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:55.500+0000 7f5e5c7c87c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:19:55.501 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:55.500+0000 7f5e5c7c87c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:19:55.501 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:55.500+0000 7f5e5c7c87c0 -1 bdev(0x55adc14adc00 td/crush-classes/2/block) open stat got: (1) Operation not permitted 2024-09-15T21:19:55.501 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:19:55.500+0000 7f5e5c7c87c0 -1 bluestore(td/crush-classes/2) _read_fsid unparsable uuid 2024-09-15T21:20:01.325 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-classes/2/keyring 2024-09-15T21:20:01.325 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:20:01.326 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2024-09-15T21:20:01.326 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd2 key to auth repository 2024-09-15T21:20:01.326 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-classes/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:20:01.584 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2024-09-15T21:20:01.584 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.2 2024-09-15T21:20:01.585 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/2 --osd-journal=td/crush-classes/2/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:20:01.585 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:20:01.585 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:20:01.586 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:20:01.617 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:01.616+0000 7fa662a517c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:20:01.620 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:01.620+0000 7fa662a517c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:20:01.621 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:01.620+0000 7fa662a517c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:20:01.825 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2024-09-15T21:20:01.826 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:20:01.826 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2024-09-15T21:20:01.826 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:20:01.826 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:20:01.826 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:01.826 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:20:01.827 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:20:01.827 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:01.827 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:02.067 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:03.068 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:03.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:03.069 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:20:03.069 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:20:03.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:03.070 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:03.138 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:03.136+0000 7fa662a517c0 -1 Falling back to public interface 2024-09-15T21:20:03.307 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:04.309 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:04.309 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:04.309 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:20:04.310 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:20:04.310 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:04.310 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:04.552 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:05.554 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:05.554 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:05.554 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:20:05.555 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:20:05.555 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:05.555 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:05.795 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:05.792+0000 7fa662a517c0 -1 osd.2 0 log_to_monitors true 2024-09-15T21:20:05.798 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:06.800 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:06.800 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:06.800 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:20:06.800 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:20:06.801 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:06.801 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:07.045 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:08.046 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:08.047 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:08.047 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:20:08.047 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:20:08.047 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:08.047 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:08.290 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:09.292 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:09.293 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:09.293 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:20:09.293 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:20:09.294 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:09.294 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:09.549 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:10.551 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:10.551 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:10.552 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:20:10.552 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:20:10.552 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:10.552 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:10.799 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:11.801 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:11.801 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:11.801 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:20:11.802 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:20:11.802 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:11.802 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:12.049 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:13.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:13.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:13.051 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:20:13.052 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:20:13.052 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:13.052 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:13.296 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:14.297 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:14.297 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:14.298 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:20:14.298 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:20:14.298 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:14.298 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:14.544 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:15.546 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:15.546 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:15.547 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:20:15.547 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:20:15.547 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:15.547 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:15.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:16.798 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:16.799 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:16.799 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:20:16.799 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:20:16.800 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:16.800 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:17.041 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:18.043 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:18.043 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:18.043 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:20:18.043 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:20:18.043 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:18.044 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:18.288 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:19.290 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:19.290 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:19.291 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:20:19.291 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:20:19.291 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:19.291 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:19.534 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:20.536 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:20.536 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:20.536 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:20:20.537 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:20:20.537 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:20.537 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:20.781 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:21.783 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:21.783 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:21.783 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:20:21.784 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:20:21.784 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:21.784 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:22.027 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:23.029 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:23.029 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:23.029 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:20:23.030 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:20:23.030 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:23.030 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:23.277 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:24.279 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:24.279 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:24.279 INFO:tasks.workunit.client.0.smithi016.stdout:18 2024-09-15T21:20:24.280 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 18 2024-09-15T21:20:24.280 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:24.280 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:24.524 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:25.525 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:25.526 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:25.526 INFO:tasks.workunit.client.0.smithi016.stdout:19 2024-09-15T21:20:25.526 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 19 2024-09-15T21:20:25.526 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:25.527 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:25.771 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:26.773 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:26.773 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:26.773 INFO:tasks.workunit.client.0.smithi016.stdout:20 2024-09-15T21:20:26.774 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 20 2024-09-15T21:20:26.774 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:26.774 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:20:27.087 INFO:tasks.workunit.client.0.smithi016.stdout:osd.2 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6816/2614649106,v1:127.0.0.1:6817/2614649106] [v2:127.0.0.1:6818/2614649106,v1:127.0.0.1:6819/2614649106] exists,up 5f9145f0-71b3-4f67-9c8b-47d461069f4f 2024-09-15T21:20:27.088 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:20:27.088 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:20:27.088 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:20:27.088 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:71: TEST_reweight_vs_classes: ceph osd crush set-device-class ssd osd.0 2024-09-15T21:20:27.724 INFO:tasks.workunit.client.0.smithi016.stderr:osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. set osd(s) to class 'ssd' 2024-09-15T21:20:27.749 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:72: TEST_reweight_vs_classes: ceph osd crush class ls-osd ssd 2024-09-15T21:20:27.749 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:72: TEST_reweight_vs_classes: grep 0 2024-09-15T21:20:28.183 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:20:28.183 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:73: TEST_reweight_vs_classes: ceph osd crush set-device-class ssd osd.1 2024-09-15T21:20:28.945 INFO:tasks.workunit.client.0.smithi016.stderr:osd.1 already set to class ssd. set-device-class item id 1 name 'osd.1' device_class 'ssd': no change. set osd(s) to class 'ssd' 2024-09-15T21:20:28.971 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:74: TEST_reweight_vs_classes: ceph osd crush class ls-osd ssd 2024-09-15T21:20:28.971 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:74: TEST_reweight_vs_classes: grep 1 2024-09-15T21:20:29.230 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:20:29.230 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:76: TEST_reweight_vs_classes: ceph osd crush reweight osd.0 1 2024-09-15T21:20:29.798 INFO:tasks.workunit.client.0.smithi016.stderr:reweighted item id 0 name 'osd.0' to 1 in crush map 2024-09-15T21:20:29.823 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:78: TEST_reweight_vs_classes: hostname -s 2024-09-15T21:20:29.825 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:78: TEST_reweight_vs_classes: h=smithi016 2024-09-15T21:20:29.825 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:79: TEST_reweight_vs_classes: ceph osd crush dump 2024-09-15T21:20:29.825 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:79: TEST_reweight_vs_classes: jq '.buckets[] | select(.name=="smithi016") | .items[0].weight' 2024-09-15T21:20:29.826 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:79: TEST_reweight_vs_classes: grep 65536 2024-09-15T21:20:30.181 INFO:tasks.workunit.client.0.smithi016.stdout:65536 2024-09-15T21:20:30.181 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:80: TEST_reweight_vs_classes: ceph osd crush dump 2024-09-15T21:20:30.181 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:80: TEST_reweight_vs_classes: jq '.buckets[] | select(.name=="smithi016~ssd") | .items[0].weight' 2024-09-15T21:20:30.181 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:80: TEST_reweight_vs_classes: grep 65536 2024-09-15T21:20:30.435 INFO:tasks.workunit.client.0.smithi016.stdout:65536 2024-09-15T21:20:30.435 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:82: TEST_reweight_vs_classes: ceph osd crush set 0 2 host=smithi016 2024-09-15T21:20:30.868 INFO:tasks.workunit.client.0.smithi016.stderr:set item id 0 name 'osd.0' weight 2 at location {host=smithi016}: no change 2024-09-15T21:20:30.890 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:84: TEST_reweight_vs_classes: ceph osd crush dump 2024-09-15T21:20:30.890 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:84: TEST_reweight_vs_classes: jq '.buckets[] | select(.name=="smithi016") | .items[0].weight' 2024-09-15T21:20:30.890 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:84: TEST_reweight_vs_classes: grep 131072 2024-09-15T21:20:31.168 INFO:tasks.workunit.client.0.smithi016.stdout:131072 2024-09-15T21:20:31.168 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:85: TEST_reweight_vs_classes: ceph osd crush dump 2024-09-15T21:20:31.169 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:85: TEST_reweight_vs_classes: grep 131072 2024-09-15T21:20:31.169 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:85: TEST_reweight_vs_classes: jq '.buckets[] | select(.name=="smithi016~ssd") | .items[0].weight' 2024-09-15T21:20:31.427 INFO:tasks.workunit.client.0.smithi016.stdout:131072 2024-09-15T21:20:31.427 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:37: run: teardown td/crush-classes 2024-09-15T21:20:31.427 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-classes 2024-09-15T21:20:31.427 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:20:31.428 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-classes KILL 2024-09-15T21:20:31.428 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:20:31.428 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:20:31.428 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:20:31.428 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:20:31.429 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:20:31.537 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:20:31.537 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:20:31.538 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:20:31.539 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:20:31.540 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:20:31.540 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:20:31.541 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:20:31.542 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:20:31.542 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:20:31.542 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:20:31.543 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:20:31.543 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:20:31.544 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:20:31.545 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-classes 2024-09-15T21:20:31.557 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:20:31.557 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:20:31.557 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:20:31.557 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39842 2024-09-15T21:20:31.559 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:20:31.559 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:20:31.559 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:34: run: for func in $funcs 2024-09-15T21:20:31.559 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:35: run: setup td/crush-classes 2024-09-15T21:20:31.559 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/crush-classes 2024-09-15T21:20:31.559 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/crush-classes 2024-09-15T21:20:31.559 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-classes 2024-09-15T21:20:31.559 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:20:31.559 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-classes KILL 2024-09-15T21:20:31.560 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:20:31.560 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:20:31.560 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:20:31.560 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:20:31.560 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:20:31.561 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:20:31.561 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:20:31.562 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:20:31.562 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:20:31.563 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:20:31.563 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:20:31.564 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:20:31.565 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:20:31.565 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:20:31.565 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:20:31.565 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:20:31.566 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:20:31.567 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:20:31.567 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-classes 2024-09-15T21:20:31.569 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:20:31.569 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:20:31.569 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:20:31.569 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39842 2024-09-15T21:20:31.570 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:20:31.570 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:20:31.570 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/crush-classes 2024-09-15T21:20:31.572 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2024-09-15T21:20:31.572 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:20:31.572 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:20:31.572 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.39842 2024-09-15T21:20:31.573 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2024-09-15T21:20:31.574 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2024-09-15T21:20:31.574 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2024-09-15T21:20:31.575 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/crush-classes 1' TERM HUP INT 2024-09-15T21:20:31.575 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:36: run: TEST_set_device_class td/crush-classes 2024-09-15T21:20:31.575 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:142: TEST_set_device_class: local dir=td/crush-classes 2024-09-15T21:20:31.575 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:144: TEST_set_device_class: TEST_classes td/crush-classes 2024-09-15T21:20:31.575 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:89: TEST_classes: local dir=td/crush-classes 2024-09-15T21:20:31.575 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:91: TEST_classes: run_mon td/crush-classes a 2024-09-15T21:20:31.575 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/crush-classes 2024-09-15T21:20:31.575 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2024-09-15T21:20:31.575 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2024-09-15T21:20:31.575 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2024-09-15T21:20:31.575 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/crush-classes/a 2024-09-15T21:20:31.575 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/crush-classes/a --run-dir=td/crush-classes 2024-09-15T21:20:32.025 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2024-09-15T21:20:32.025 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:20:32.025 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:20:32.025 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:20:32.025 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:20:32.025 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:20:32.026 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:20:32.026 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/crush-classes/a '--log-file=td/crush-classes/$name.log' '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --mon-cluster-log-file=td/crush-classes/log --run-dir=td/crush-classes '--pid-file=td/crush-classes/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2024-09-15T21:20:32.212 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2024-09-15T21:20:32.214 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2024-09-15T21:20:32.214 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:20:32.214 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:20:32.214 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2024-09-15T21:20:32.214 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2024-09-15T21:20:32.215 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:20:32.215 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:20:32.215 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:20:32.216 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:20:32.216 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:20:32.216 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:20:32.216 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39842/ceph-mon.a.asok 2024-09-15T21:20:32.216 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:20:32.216 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39842/ceph-mon.a.asok config get fsid 2024-09-15T21:20:32.336 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2024-09-15T21:20:32.336 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2024-09-15T21:20:32.336 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2024-09-15T21:20:32.336 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2024-09-15T21:20:32.337 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2024-09-15T21:20:32.337 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2024-09-15T21:20:32.337 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2024-09-15T21:20:32.337 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2024-09-15T21:20:32.337 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2024-09-15T21:20:32.338 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:20:32.338 INFO:tasks.workunit.client.0.smithi016.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:20:32.338 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39842/ceph-mon.a.asok 2024-09-15T21:20:32.338 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2024-09-15T21:20:32.338 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39842/ceph-mon.a.asok config get mon_host 2024-09-15T21:20:32.455 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:92: TEST_classes: run_osd td/crush-classes 0 2024-09-15T21:20:32.455 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-classes 2024-09-15T21:20:32.455 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:20:32.455 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2024-09-15T21:20:32.455 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:20:32.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-classes/0 2024-09-15T21:20:32.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false ' 2024-09-15T21:20:32.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:20:32.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:20:32.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:20:32.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-classes/0' 2024-09-15T21:20:32.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-classes/0/journal' 2024-09-15T21:20:32.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:20:32.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:20:32.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-classes' 2024-09-15T21:20:32.456 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:20:32.456 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:20:32.456 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:20:32.457 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:20:32.457 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:20:32.457 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:20:32.457 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:20:32.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:20:32.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:20:32.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:20:32.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:20:32.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-classes/$name.log' 2024-09-15T21:20:32.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-classes/$name.pid' 2024-09-15T21:20:32.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:20:32.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:20:32.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:20:32.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:20:32.458 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:20:32.458 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:20:32.458 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-classes/0 2024-09-15T21:20:32.458 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:20:32.459 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8f855884-6ffd-4696-ba1a-d2fb61541f1a 2024-09-15T21:20:32.459 INFO:tasks.workunit.client.0.smithi016.stdout:add osd0 8f855884-6ffd-4696-ba1a-d2fb61541f1a 2024-09-15T21:20:32.459 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8f855884-6ffd-4696-ba1a-d2fb61541f1a' 2024-09-15T21:20:32.460 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:20:32.483 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCgT+dm9UzWHBAAOTmxjGCIjQs5FqOnaw/ChQ== 2024-09-15T21:20:32.483 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCgT+dm9UzWHBAAOTmxjGCIjQs5FqOnaw/ChQ=="}' 2024-09-15T21:20:32.483 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8f855884-6ffd-4696-ba1a-d2fb61541f1a -i td/crush-classes/0/new.json 2024-09-15T21:20:33.009 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:20:33.026 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-classes/0/new.json 2024-09-15T21:20:33.028 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/0 --osd-journal=td/crush-classes/0/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCgT+dm9UzWHBAAOTmxjGCIjQs5FqOnaw/ChQ== --osd-uuid 8f855884-6ffd-4696-ba1a-d2fb61541f1a 2024-09-15T21:20:33.070 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:33.067+0000 7f7169bdb7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:20:33.073 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:33.071+0000 7f7169bdb7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:20:33.075 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:33.071+0000 7f7169bdb7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:20:33.075 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:33.071+0000 7f7169bdb7c0 -1 bdev(0x559438ef1c00 td/crush-classes/0/block) open stat got: (1) Operation not permitted 2024-09-15T21:20:33.075 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:33.071+0000 7f7169bdb7c0 -1 bluestore(td/crush-classes/0) _read_fsid unparsable uuid 2024-09-15T21:20:38.371 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-classes/0/keyring 2024-09-15T21:20:38.371 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:20:38.372 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd0 key to auth repository 2024-09-15T21:20:38.373 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2024-09-15T21:20:38.373 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-classes/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:20:38.634 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.0 2024-09-15T21:20:38.634 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2024-09-15T21:20:38.634 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/0 --osd-journal=td/crush-classes/0/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:20:38.635 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:20:38.635 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:20:38.635 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:20:38.667 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:38.663+0000 7f71fd0a17c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:20:38.669 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:38.667+0000 7f71fd0a17c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:20:38.671 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:38.667+0000 7f71fd0a17c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:20:38.872 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2024-09-15T21:20:38.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:20:38.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2024-09-15T21:20:38.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:20:38.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:20:38.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:38.873 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:20:38.874 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:20:38.874 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:38.874 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:39.105 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:39.911 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:39.907+0000 7f71fd0a17c0 -1 Falling back to public interface 2024-09-15T21:20:40.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:40.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:40.108 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:20:40.108 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:20:40.109 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:40.109 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:40.351 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:41.352 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:41.353 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:41.353 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:20:41.353 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:20:41.353 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:41.353 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:41.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:42.574 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:20:42.571+0000 7f71fd0a17c0 -1 osd.0 0 log_to_monitors true 2024-09-15T21:20:42.590 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:42.591 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:20:42.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:42.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:20:42.591 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:42.592 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:43.077 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:44.079 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:44.079 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:44.079 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:20:44.079 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:20:44.080 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:44.080 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:44.324 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:45.325 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:45.325 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:45.326 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:20:45.326 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:20:45.326 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:45.326 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:45.570 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:46.571 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:46.572 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:46.572 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:20:46.572 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:20:46.572 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:46.573 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:46.815 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:47.817 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:47.817 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:47.817 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:20:47.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:20:47.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:47.818 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:48.064 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:49.065 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:49.065 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:49.065 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:20:49.066 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:20:49.066 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:49.066 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:49.311 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:50.313 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:50.313 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:50.314 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:20:50.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:20:50.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:50.314 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:50.560 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:51.562 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:51.562 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:51.562 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:20:51.563 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:20:51.563 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:51.563 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:51.807 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:52.808 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:52.808 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:52.809 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:20:52.809 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:20:52.809 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:52.809 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:53.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:54.053 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:54.053 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:54.054 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:20:54.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:20:54.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:54.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:54.299 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:55.301 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:55.301 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:55.302 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:20:55.302 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:20:55.302 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:55.302 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:55.550 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:56.551 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:56.552 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:56.552 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:20:56.552 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:20:56.553 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:56.553 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:56.796 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:57.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:57.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:57.798 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:20:57.798 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:20:57.798 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:57.798 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:58.043 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:20:59.045 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:20:59.045 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:20:59.046 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:20:59.046 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:20:59.046 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:20:59.046 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:20:59.298 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:00.299 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:00.299 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:00.299 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:21:00.300 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:21:00.300 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:00.300 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:21:00.546 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:01.548 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:01.549 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:01.549 INFO:tasks.workunit.client.0.smithi016.stdout:18 2024-09-15T21:21:01.550 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 18 2024-09-15T21:21:01.550 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:01.550 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:21:01.794 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:02.796 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:02.796 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:02.796 INFO:tasks.workunit.client.0.smithi016.stdout:19 2024-09-15T21:21:02.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 19 2024-09-15T21:21:02.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:02.797 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2024-09-15T21:21:03.047 INFO:tasks.workunit.client.0.smithi016.stdout:osd.0 up in weight 1 up_from 4 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4154301310,v1:127.0.0.1:6801/4154301310] [v2:127.0.0.1:6802/4154301310,v1:127.0.0.1:6803/4154301310] exists,up 8f855884-6ffd-4696-ba1a-d2fb61541f1a 2024-09-15T21:21:03.050 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:21:03.050 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:21:03.050 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:21:03.050 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:93: TEST_classes: run_osd td/crush-classes 1 2024-09-15T21:21:03.050 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-classes 2024-09-15T21:21:03.050 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:21:03.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2024-09-15T21:21:03.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:21:03.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-classes/1 2024-09-15T21:21:03.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false ' 2024-09-15T21:21:03.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:21:03.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:21:03.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:21:03.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-classes/1' 2024-09-15T21:21:03.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-classes/1/journal' 2024-09-15T21:21:03.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:21:03.052 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:21:03.052 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-classes' 2024-09-15T21:21:03.052 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:21:03.052 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:21:03.052 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:21:03.052 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:21:03.052 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:21:03.052 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:21:03.052 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:21:03.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:21:03.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:21:03.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:21:03.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:21:03.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-classes/$name.log' 2024-09-15T21:21:03.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-classes/$name.pid' 2024-09-15T21:21:03.054 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:21:03.055 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:21:03.055 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:21:03.055 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:21:03.055 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:21:03.055 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:21:03.055 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-classes/1 2024-09-15T21:21:03.055 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:21:03.055 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6d9e2734-4dbe-47a0-8ffe-fbab085c49e6 2024-09-15T21:21:03.055 INFO:tasks.workunit.client.0.smithi016.stdout:add osd1 6d9e2734-4dbe-47a0-8ffe-fbab085c49e6 2024-09-15T21:21:03.056 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 6d9e2734-4dbe-47a0-8ffe-fbab085c49e6' 2024-09-15T21:21:03.056 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:21:03.077 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC/T+dmIhaXBBAAALQcvh29n5tLrJqfzVonwA== 2024-09-15T21:21:03.077 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC/T+dmIhaXBBAAALQcvh29n5tLrJqfzVonwA=="}' 2024-09-15T21:21:03.077 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6d9e2734-4dbe-47a0-8ffe-fbab085c49e6 -i td/crush-classes/1/new.json 2024-09-15T21:21:03.410 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:21:03.427 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-classes/1/new.json 2024-09-15T21:21:03.429 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/1 --osd-journal=td/crush-classes/1/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC/T+dmIhaXBBAAALQcvh29n5tLrJqfzVonwA== --osd-uuid 6d9e2734-4dbe-47a0-8ffe-fbab085c49e6 2024-09-15T21:21:03.466 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:03.463+0000 7f2b075657c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:21:03.497 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:03.495+0000 7f2b075657c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:21:03.498 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:03.495+0000 7f2b075657c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:21:03.499 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:03.495+0000 7f2b075657c0 -1 bdev(0x561f05429c00 td/crush-classes/1/block) open stat got: (1) Operation not permitted 2024-09-15T21:21:03.499 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:03.495+0000 7f2b075657c0 -1 bluestore(td/crush-classes/1) _read_fsid unparsable uuid 2024-09-15T21:21:09.034 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-classes/1/keyring 2024-09-15T21:21:09.034 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:21:09.036 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd1 key to auth repository 2024-09-15T21:21:09.036 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2024-09-15T21:21:09.036 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-classes/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:21:09.302 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.1 2024-09-15T21:21:09.302 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2024-09-15T21:21:09.302 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/1 --osd-journal=td/crush-classes/1/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:21:09.303 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:21:09.303 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:21:09.303 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:21:09.336 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:09.331+0000 7fc59472f7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:21:09.338 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:09.335+0000 7fc59472f7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:21:09.339 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:09.335+0000 7fc59472f7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:21:09.538 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2024-09-15T21:21:09.538 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:21:09.538 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2024-09-15T21:21:09.538 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:21:09.538 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:21:09.538 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:09.538 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:21:09.539 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:21:09.539 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:09.539 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:09.777 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:10.627 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:10.623+0000 7fc59472f7c0 -1 Falling back to public interface 2024-09-15T21:21:10.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:10.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:10.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:21:10.779 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:21:10.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:10.780 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:11.021 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:12.022 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:12.023 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:12.023 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:21:12.023 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:21:12.023 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:12.023 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:12.266 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:13.217 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:13.215+0000 7fc59472f7c0 -1 osd.1 0 log_to_monitors true 2024-09-15T21:21:13.267 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:13.267 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:13.267 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:21:13.268 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:21:13.268 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:13.268 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:13.728 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:14.730 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:14.731 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:14.731 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:21:14.731 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:21:14.731 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:14.732 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:14.977 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:15.979 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:15.979 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:15.979 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:21:15.979 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:21:15.980 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:15.980 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:16.221 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:17.223 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:17.223 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:17.224 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:21:17.224 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:21:17.224 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:17.224 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:17.467 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:18.469 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:18.470 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:18.470 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:21:18.470 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:21:18.471 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:18.471 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:18.712 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:19.714 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:19.714 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:19.714 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:21:19.715 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:21:19.715 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:19.715 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:19.963 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:20.965 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:20.965 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:20.965 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:21:20.966 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:21:20.966 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:20.966 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:21.212 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:22.213 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:22.214 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:22.214 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:21:22.214 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:21:22.214 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:22.214 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:22.454 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:23.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:23.456 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:23.456 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:21:23.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:21:23.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:23.457 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:23.697 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:24.698 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:24.698 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:24.699 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:21:24.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:21:24.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:24.699 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:24.941 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:25.942 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:25.942 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:25.943 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:21:25.943 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:21:25.943 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:25.944 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:26.183 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:27.185 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:27.185 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:27.185 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:21:27.186 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:21:27.186 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:27.186 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:27.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:28.435 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:28.435 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:28.436 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:21:28.436 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:21:28.436 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:28.436 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:28.678 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:29.680 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:29.680 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:29.681 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:21:29.681 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:21:29.681 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:29.681 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:29.928 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:30.930 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:30.930 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:30.931 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:21:30.931 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:21:30.931 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:30.931 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:31.175 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:32.177 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:32.177 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:32.177 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 18 2024-09-15T21:21:32.178 INFO:tasks.workunit.client.0.smithi016.stdout:18 2024-09-15T21:21:32.178 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:32.178 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:32.415 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:33.417 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:33.417 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:33.417 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 19 2024-09-15T21:21:33.418 INFO:tasks.workunit.client.0.smithi016.stdout:19 2024-09-15T21:21:33.418 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:33.418 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2024-09-15T21:21:33.666 INFO:tasks.workunit.client.0.smithi016.stdout:osd.1 up in weight 1 up_from 8 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6808/3962745450,v1:127.0.0.1:6809/3962745450] [v2:127.0.0.1:6810/3962745450,v1:127.0.0.1:6811/3962745450] exists,up 6d9e2734-4dbe-47a0-8ffe-fbab085c49e6 2024-09-15T21:21:33.669 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:21:33.670 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:21:33.670 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:21:33.670 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:94: TEST_classes: run_osd td/crush-classes 2 2024-09-15T21:21:33.670 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/crush-classes 2024-09-15T21:21:33.670 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2024-09-15T21:21:33.670 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2024-09-15T21:21:33.670 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2024-09-15T21:21:33.670 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/crush-classes/2 2024-09-15T21:21:33.670 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false ' 2024-09-15T21:21:33.670 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2024-09-15T21:21:33.670 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2024-09-15T21:21:33.670 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2024-09-15T21:21:33.670 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/crush-classes/2' 2024-09-15T21:21:33.671 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/crush-classes/2/journal' 2024-09-15T21:21:33.671 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2024-09-15T21:21:33.671 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2024-09-15T21:21:33.671 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/crush-classes' 2024-09-15T21:21:33.671 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2024-09-15T21:21:33.671 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2024-09-15T21:21:33.671 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2024-09-15T21:21:33.671 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2024-09-15T21:21:33.671 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:21:33.671 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:21:33.671 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:21:33.671 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' 2024-09-15T21:21:33.671 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2024-09-15T21:21:33.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2024-09-15T21:21:33.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2024-09-15T21:21:33.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/crush-classes/$name.log' 2024-09-15T21:21:33.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/crush-classes/$name.pid' 2024-09-15T21:21:33.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2024-09-15T21:21:33.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2024-09-15T21:21:33.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2024-09-15T21:21:33.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2024-09-15T21:21:33.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2024-09-15T21:21:33.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2024-09-15T21:21:33.672 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/crush-classes/2 2024-09-15T21:21:33.673 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2024-09-15T21:21:33.673 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dca4c4bd-1a01-44d4-92d0-f0b29e5033e3 2024-09-15T21:21:33.673 INFO:tasks.workunit.client.0.smithi016.stdout:add osd2 dca4c4bd-1a01-44d4-92d0-f0b29e5033e3 2024-09-15T21:21:33.674 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 dca4c4bd-1a01-44d4-92d0-f0b29e5033e3' 2024-09-15T21:21:33.674 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2024-09-15T21:21:33.690 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDdT+dmzeYuKRAAZM+8iZuvjdrn7x4ZwqkLXg== 2024-09-15T21:21:33.691 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDdT+dmzeYuKRAAZM+8iZuvjdrn7x4ZwqkLXg=="}' 2024-09-15T21:21:33.691 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dca4c4bd-1a01-44d4-92d0-f0b29e5033e3 -i td/crush-classes/2/new.json 2024-09-15T21:21:34.033 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:21:34.050 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/crush-classes/2/new.json 2024-09-15T21:21:34.052 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/2 --osd-journal=td/crush-classes/2/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDdT+dmzeYuKRAAZM+8iZuvjdrn7x4ZwqkLXg== --osd-uuid dca4c4bd-1a01-44d4-92d0-f0b29e5033e3 2024-09-15T21:21:34.081 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:34.078+0000 7fe0c6baa7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:21:34.152 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:34.150+0000 7fe0c6baa7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:21:34.154 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:34.150+0000 7fe0c6baa7c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:21:34.154 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:34.150+0000 7fe0c6baa7c0 -1 bdev(0x5620e4839c00 td/crush-classes/2/block) open stat got: (1) Operation not permitted 2024-09-15T21:21:34.155 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:34.150+0000 7fe0c6baa7c0 -1 bluestore(td/crush-classes/2) _read_fsid unparsable uuid 2024-09-15T21:21:40.013 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/crush-classes/2/keyring 2024-09-15T21:21:40.013 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2024-09-15T21:21:40.014 INFO:tasks.workunit.client.0.smithi016.stdout:adding osd2 key to auth repository 2024-09-15T21:21:40.014 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2024-09-15T21:21:40.014 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/crush-classes/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2024-09-15T21:21:40.273 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2024-09-15T21:21:40.274 INFO:tasks.workunit.client.0.smithi016.stdout:start osd.2 2024-09-15T21:21:40.274 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=35e511f5-16a4-4741-aa4c-2c5cb1d5602b --auth-supported=none --mon-host=127.0.0.1:7130 --osd-class-update-on-start=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/crush-classes/2 --osd-journal=td/crush-classes/2/journal --chdir= --run-dir=td/crush-classes '--admin-socket=/tmp/ceph-asok.39842/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/crush-classes/$name.log' '--pid-file=td/crush-classes/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2024-09-15T21:21:40.274 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2024-09-15T21:21:40.275 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2024-09-15T21:21:40.275 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2024-09-15T21:21:40.301 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:40.298+0000 7fc1ccde57c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:21:40.303 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:40.298+0000 7fc1ccde57c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:21:40.305 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:40.302+0000 7fc1ccde57c0 -1 WARNING: all dangerous and experimental features are enabled. 2024-09-15T21:21:40.511 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2024-09-15T21:21:40.511 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2024-09-15T21:21:40.511 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2024-09-15T21:21:40.511 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2024-09-15T21:21:40.511 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2024-09-15T21:21:40.511 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:40.511 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:21:40.512 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2024-09-15T21:21:40.512 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:40.512 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:40.744 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:41.575 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:41.570+0000 7fc1ccde57c0 -1 Falling back to public interface 2024-09-15T21:21:41.748 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:41.749 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:41.749 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:21:41.750 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2024-09-15T21:21:41.751 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:41.751 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:41.990 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:42.992 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:42.992 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:42.993 INFO:tasks.workunit.client.0.smithi016.stdout:2 2024-09-15T21:21:42.993 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2024-09-15T21:21:42.994 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:42.994 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:43.237 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:44.239 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:44.239 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:44.239 INFO:tasks.workunit.client.0.smithi016.stdout:3 2024-09-15T21:21:44.240 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2024-09-15T21:21:44.240 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:44.240 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:44.246 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-15T21:21:44.242+0000 7fc1ccde57c0 -1 osd.2 0 log_to_monitors true 2024-09-15T21:21:44.777 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:45.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:45.779 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:45.779 INFO:tasks.workunit.client.0.smithi016.stdout:4 2024-09-15T21:21:45.780 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2024-09-15T21:21:45.780 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:45.780 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:46.024 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:47.026 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:47.026 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:47.026 INFO:tasks.workunit.client.0.smithi016.stdout:5 2024-09-15T21:21:47.027 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2024-09-15T21:21:47.027 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:47.027 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:47.275 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:48.278 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:48.278 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:48.279 INFO:tasks.workunit.client.0.smithi016.stdout:6 2024-09-15T21:21:48.279 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2024-09-15T21:21:48.279 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:48.279 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:48.521 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:49.523 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:49.523 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:49.523 INFO:tasks.workunit.client.0.smithi016.stdout:7 2024-09-15T21:21:49.524 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2024-09-15T21:21:49.524 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:49.524 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:49.762 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:50.763 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:50.763 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:50.764 INFO:tasks.workunit.client.0.smithi016.stdout:8 2024-09-15T21:21:50.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2024-09-15T21:21:50.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:50.764 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:51.010 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:52.012 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:52.012 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:52.012 INFO:tasks.workunit.client.0.smithi016.stdout:9 2024-09-15T21:21:52.013 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2024-09-15T21:21:52.013 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:52.013 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:52.254 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:53.255 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:53.256 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:53.256 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2024-09-15T21:21:53.256 INFO:tasks.workunit.client.0.smithi016.stdout:10 2024-09-15T21:21:53.256 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:53.256 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:53.496 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:54.498 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:54.499 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:54.499 INFO:tasks.workunit.client.0.smithi016.stdout:11 2024-09-15T21:21:54.499 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2024-09-15T21:21:54.499 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:54.500 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:54.740 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:55.742 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:55.742 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:55.742 INFO:tasks.workunit.client.0.smithi016.stdout:12 2024-09-15T21:21:55.743 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2024-09-15T21:21:55.743 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:55.743 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:55.990 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:56.991 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:56.991 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:56.992 INFO:tasks.workunit.client.0.smithi016.stdout:13 2024-09-15T21:21:56.992 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 13 2024-09-15T21:21:56.992 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:56.992 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:57.230 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:58.232 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:58.232 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:58.233 INFO:tasks.workunit.client.0.smithi016.stdout:14 2024-09-15T21:21:58.233 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 14 2024-09-15T21:21:58.233 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:58.233 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:58.474 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:21:59.476 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:21:59.476 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:21:59.476 INFO:tasks.workunit.client.0.smithi016.stdout:15 2024-09-15T21:21:59.477 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 15 2024-09-15T21:21:59.477 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:21:59.477 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:21:59.720 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:22:00.721 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:22:00.722 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:22:00.722 INFO:tasks.workunit.client.0.smithi016.stdout:16 2024-09-15T21:22:00.722 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 16 2024-09-15T21:22:00.722 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:22:00.722 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:22:00.967 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:22:01.969 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:22:01.969 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:22:01.969 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 17 2024-09-15T21:22:01.970 INFO:tasks.workunit.client.0.smithi016.stdout:17 2024-09-15T21:22:01.970 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:22:01.970 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:22:02.215 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:22:03.217 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:22:03.217 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:22:03.218 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 18 2024-09-15T21:22:03.218 INFO:tasks.workunit.client.0.smithi016.stdout:18 2024-09-15T21:22:03.218 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:22:03.218 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:22:03.462 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:22:04.464 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:22:04.465 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:22:04.465 INFO:tasks.workunit.client.0.smithi016.stdout:19 2024-09-15T21:22:04.466 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 19 2024-09-15T21:22:04.466 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:22:04.466 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:22:04.711 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2024-09-15T21:22:05.713 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2024-09-15T21:22:05.713 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2024-09-15T21:22:05.713 INFO:tasks.workunit.client.0.smithi016.stdout:20 2024-09-15T21:22:05.714 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 20 2024-09-15T21:22:05.714 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2024-09-15T21:22:05.714 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2024-09-15T21:22:05.956 INFO:tasks.workunit.client.0.smithi016.stdout:osd.2 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6816/2865131932,v1:127.0.0.1:6817/2865131932] [v2:127.0.0.1:6818/2865131932,v1:127.0.0.1:6819/2865131932] exists,up dca4c4bd-1a01-44d4-92d0-f0b29e5033e3 2024-09-15T21:22:05.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2024-09-15T21:22:05.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2024-09-15T21:22:05.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2024-09-15T21:22:05.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:95: TEST_classes: create_rbd_pool 2024-09-15T21:22:05.957 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2024-09-15T21:22:06.172 INFO:tasks.workunit.client.0.smithi016.stderr:pool 'rbd' does not exist 2024-09-15T21:22:06.187 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2024-09-15T21:22:06.187 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2024-09-15T21:22:06.570 INFO:tasks.workunit.client.0.smithi016.stderr:pool 'rbd' already exists 2024-09-15T21:22:06.590 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2024-09-15T21:22:07.592 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2024-09-15T21:22:09.158 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:97: TEST_classes: get_osds_up rbd SOMETHING 2024-09-15T21:22:09.158 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:51: get_osds_up: local poolname=rbd 2024-09-15T21:22:09.158 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:52: get_osds_up: local objectname=SOMETHING 2024-09-15T21:22:09.159 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: ceph --format xml osd map rbd SOMETHING 2024-09-15T21:22:09.159 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: xmlstarlet sel -t -m //up/osd -v . -o ' ' 2024-09-15T21:22:09.432 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: local 'osds=1 2 0 ' 2024-09-15T21:22:09.433 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:57: get_osds_up: echo 1 2 0 2024-09-15T21:22:09.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:97: TEST_classes: test '1 2 0' == '1 2 0' 2024-09-15T21:22:09.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:98: TEST_classes: add_something td/crush-classes SOMETHING 2024-09-15T21:22:09.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:42: add_something: local dir=td/crush-classes 2024-09-15T21:22:09.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:43: add_something: local obj=SOMETHING 2024-09-15T21:22:09.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:45: add_something: local payload=ABCDEF 2024-09-15T21:22:09.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:46: add_something: echo ABCDEF 2024-09-15T21:22:09.433 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:47: add_something: rados --pool rbd put SOMETHING td/crush-classes/ORIGINAL 2024-09-15T21:22:09.605 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:104: TEST_classes: ceph osd getcrushmap 2024-09-15T21:22:09.836 INFO:tasks.workunit.client.0.smithi016.stderr:4 2024-09-15T21:22:09.851 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:105: TEST_classes: crushtool -d td/crush-classes/map -o td/crush-classes/map.txt 2024-09-15T21:22:09.873 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:106: TEST_classes: sed -i -e '/device 0 osd.0/s/$/ class ssd/' -e '/step take default/s/$/ class ssd/' td/crush-classes/map.txt 2024-09-15T21:22:09.875 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:110: TEST_classes: crushtool -c td/crush-classes/map.txt -o td/crush-classes/map-new 2024-09-15T21:22:09.892 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:111: TEST_classes: ceph osd setcrushmap -i td/crush-classes/map-new 2024-09-15T21:22:10.597 INFO:tasks.workunit.client.0.smithi016.stderr:6 2024-09-15T21:22:10.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:117: TEST_classes: ok=false 2024-09-15T21:22:10.618 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:118: TEST_classes: for delay in 2 4 8 16 32 64 128 256 2024-09-15T21:22:10.618 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:119: TEST_classes: get_osds_up rbd SOMETHING_ELSE 2024-09-15T21:22:10.619 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:51: get_osds_up: local poolname=rbd 2024-09-15T21:22:10.619 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:52: get_osds_up: local objectname=SOMETHING_ELSE 2024-09-15T21:22:10.620 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: ceph --format xml osd map rbd SOMETHING_ELSE 2024-09-15T21:22:10.620 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: xmlstarlet sel -t -m //up/osd -v . -o ' ' 2024-09-15T21:22:11.023 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: local 'osds=0 ' 2024-09-15T21:22:11.023 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:57: get_osds_up: echo 0 2024-09-15T21:22:11.024 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:119: TEST_classes: test 0 == 0 2024-09-15T21:22:11.024 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:120: TEST_classes: ok=true 2024-09-15T21:22:11.024 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:121: TEST_classes: break 2024-09-15T21:22:11.024 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:127: TEST_classes: true 2024-09-15T21:22:11.024 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:132: TEST_classes: add_something td/crush-classes SOMETHING_ELSE 2024-09-15T21:22:11.025 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:42: add_something: local dir=td/crush-classes 2024-09-15T21:22:11.025 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:43: add_something: local obj=SOMETHING_ELSE 2024-09-15T21:22:11.025 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:45: add_something: local payload=ABCDEF 2024-09-15T21:22:11.025 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:46: add_something: echo ABCDEF 2024-09-15T21:22:11.025 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:47: add_something: rados --pool rbd put SOMETHING_ELSE td/crush-classes/ORIGINAL 2024-09-15T21:22:12.204 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:138: TEST_classes: ceph osd crush dump 2024-09-15T21:22:12.204 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:138: TEST_classes: grep -q '~ssd' 2024-09-15T21:22:12.452 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:146: TEST_set_device_class: ceph osd crush set-device-class ssd osd.0 2024-09-15T21:22:12.925 INFO:tasks.workunit.client.0.smithi016.stderr:osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. set osd(s) to class 'ssd' 2024-09-15T21:22:12.948 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:147: TEST_set_device_class: ceph osd crush class ls-osd ssd 2024-09-15T21:22:12.948 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:147: TEST_set_device_class: grep 0 2024-09-15T21:22:13.245 INFO:tasks.workunit.client.0.smithi016.stdout:0 2024-09-15T21:22:13.245 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:148: TEST_set_device_class: ceph osd crush set-device-class ssd osd.1 2024-09-15T21:22:14.014 INFO:tasks.workunit.client.0.smithi016.stderr:osd.1 already set to class ssd. set-device-class item id 1 name 'osd.1' device_class 'ssd': no change. set osd(s) to class 'ssd' 2024-09-15T21:22:14.044 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:149: TEST_set_device_class: ceph osd crush class ls-osd ssd 2024-09-15T21:22:14.044 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:149: TEST_set_device_class: grep 1 2024-09-15T21:22:14.414 INFO:tasks.workunit.client.0.smithi016.stdout:1 2024-09-15T21:22:14.414 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:150: TEST_set_device_class: ceph osd crush set-device-class ssd 0 1 2024-09-15T21:22:15.575 INFO:tasks.workunit.client.0.smithi016.stderr:osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. osd.1 already set to class ssd. set-device-class item id 1 name 'osd.1' device_class 'ssd': no change. set osd(s) to class 'ssd' 2024-09-15T21:22:15.604 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:152: TEST_set_device_class: ok=false 2024-09-15T21:22:15.604 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:153: TEST_set_device_class: for delay in 2 4 8 16 32 64 128 256 2024-09-15T21:22:15.605 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:154: TEST_set_device_class: get_osds_up rbd SOMETHING_ELSE 2024-09-15T21:22:15.605 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:51: get_osds_up: local poolname=rbd 2024-09-15T21:22:15.605 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:52: get_osds_up: local objectname=SOMETHING_ELSE 2024-09-15T21:22:15.606 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: ceph --format xml osd map rbd SOMETHING_ELSE 2024-09-15T21:22:15.606 INFO:tasks.workunit.client.0.smithi016.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: xmlstarlet sel -t -m //up/osd -v . -o ' ' 2024-09-15T21:22:15.906 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:55: get_osds_up: local 'osds=0 1 ' 2024-09-15T21:22:15.906 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:57: get_osds_up: echo 0 1 2024-09-15T21:22:15.906 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:154: TEST_set_device_class: test '0 1' == '0 1' 2024-09-15T21:22:15.906 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:155: TEST_set_device_class: ok=true 2024-09-15T21:22:15.906 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:156: TEST_set_device_class: break 2024-09-15T21:22:15.906 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:163: TEST_set_device_class: true 2024-09-15T21:22:15.906 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-classes.sh:37: run: teardown td/crush-classes 2024-09-15T21:22:15.906 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-classes 2024-09-15T21:22:15.907 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2024-09-15T21:22:15.907 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-classes KILL 2024-09-15T21:22:15.907 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:22:15.907 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:22:15.907 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:22:15.907 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:22:15.908 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:22:16.015 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:22:16.016 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:22:16.017 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:22:16.017 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:22:16.018 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:22:16.018 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:22:16.018 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:22:16.019 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:22:16.019 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:22:16.020 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:22:16.020 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:22:16.021 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:22:16.022 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2024-09-15T21:22:16.022 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-classes 2024-09-15T21:22:16.036 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:22:16.036 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:22:16.036 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:22:16.036 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39842 2024-09-15T21:22:16.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:22:16.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:22:16.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2322: main: code=0 2024-09-15T21:22:16.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2326: main: teardown td/crush-classes 0 2024-09-15T21:22:16.039 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/crush-classes 2024-09-15T21:22:16.040 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2024-09-15T21:22:16.040 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/crush-classes KILL 2024-09-15T21:22:16.040 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2024-09-15T21:22:16.040 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2024-09-15T21:22:16.040 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2024-09-15T21:22:16.041 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2024-09-15T21:22:16.041 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2024-09-15T21:22:16.042 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2024-09-15T21:22:16.042 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2024-09-15T21:22:16.042 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2024-09-15T21:22:16.043 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2024-09-15T21:22:16.043 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2024-09-15T21:22:16.043 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2024-09-15T21:22:16.044 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2024-09-15T21:22:16.044 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:22:16.045 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2024-09-15T21:22:16.045 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2024-09-15T21:22:16.045 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:22:16.046 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:22:16.046 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2024-09-15T21:22:16.046 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/crush-classes 2024-09-15T21:22:16.048 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2024-09-15T21:22:16.048 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2024-09-15T21:22:16.048 INFO:tasks.workunit.client.0.smithi016.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39842 2024-09-15T21:22:16.048 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39842 2024-09-15T21:22:16.050 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2024-09-15T21:22:16.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2024-09-15T21:22:16.051 INFO:tasks.workunit.client.0.smithi016.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2327: main: return 0 2024-09-15T21:22:16.051 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T21:22:16.052 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T21:22:16.062 INFO:tasks.workunit:Stopping ['crush'] on client.0... 2024-09-15T21:22:16.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-15T21:22:16.417 DEBUG:teuthology.parallel:result is None 2024-09-15T21:22:16.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T21:22:16.429 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T21:22:16.429 DEBUG:teuthology.orchestra.run.smithi016:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-15T21:22:16.488 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T21:22:16.489 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T21:22:16.498 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T21:22:16.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T21:22:16.570 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T21:22:16.579 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T21:22:16.580 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:22:16.596 INFO:teuthology.orchestra.run.smithi016.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T21:22:16.596 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================== 2024-09-15T21:22:16.596 INFO:teuthology.orchestra.run.smithi016.stdout:+hv01.front.sepi 67.205.162.81 3 u 55 64 377 0.082 -5.416 0.673 2024-09-15T21:22:16.596 INFO:teuthology.orchestra.run.smithi016.stdout:*hv02.front.sepi 63.231.80.2 3 u 56 64 377 0.109 -4.183 0.571 2024-09-15T21:22:16.596 INFO:teuthology.orchestra.run.smithi016.stdout:+hv03.front.sepi 74.6.168.72 3 u 56 64 377 0.098 -1.509 1.008 2024-09-15T21:22:16.596 INFO:teuthology.orchestra.run.smithi016.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-15T21:22:16.597 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T21:22:16.605 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T21:22:16.606 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T21:22:16.615 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T21:22:16.627 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T21:22:16.636 INFO:teuthology.task.internal:Duration was 1440.990210 seconds 2024-09-15T21:22:16.637 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T21:22:16.644 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T21:22:16.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T21:22:16.685 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T21:22:16.685 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-09-15T21:22:16.686 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T21:22:16.736 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T21:22:16.737 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T21:22:16.817 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T21:22:16.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T21:22:16.914 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T21:22:16.924 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T21:22:16.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T21:22:16.935 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T21:22:16.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:22:16.985 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-09-15T21:22:16.997 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:22:17.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:22:17.048 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T21:22:17.058 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T21:22:17.060 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905658/remote/smithi016 2024-09-15T21:22:17.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T21:22:17.111 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T21:22:17.111 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T21:22:17.154 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T21:22:17.171 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T21:22:17.171 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T21:22:17.185 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T21:22:17.188 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T21:22:17.201 INFO:teuthology.orchestra.run.smithi016.stdout: 658641 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 21:22 /home/ubuntu/cephtest 2024-09-15T21:22:17.203 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T21:22:17.213 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T21:22:17.238 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/crush} duration: 1440.9902102947235 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T21:22:17.238 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:22:17.324 INFO:teuthology.run:pass