2023-12-01T06:09:02.595 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-01T06:09:02.595 DEBUG:teuthology.run:Teuthology command: teuthology --name rfriedma-2023-11-30_18:48:32-rados-wip-rf-repl-hp-distro-default-smithi --verbose --owner scheduled_rfriedma@teuthology --description rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} --archive /home/teuthworker/archive/rfriedma-2023-11-30_18:48:32-rados-wip-rf-repl-hp-distro-default-smithi/7472795 -- /home/teuthworker/archive/rfriedma-2023-11-30_18:48:32-rados-wip-rf-repl-hp-distro-default-smithi/7472795/orig.config.yaml 2023-12-01T06:09:02.620 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-01T06:09:02.707 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rfriedma-2023-11-30_18:48:32-rados-wip-rf-repl-hp-distro-default-smithi/7472795 branch: wip-rf-repl-hp description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} email: null first_in_suite: false job_id: '7472795' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: rfriedma-2023-11-30_18:48:32-rados-wip-rf-repl-hp-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-rf-repl-hp ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: bbeb098e76b9019fafbe1c7bd55e48604c324c89 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: bbeb098e76b9019fafbe1c7bd55e48604c324c89 workunit: branch: wip-rf-repl-hp sha1: bbeb098e76b9019fafbe1c7bd55e48604c324c89 owner: scheduled_rfriedma@teuthology priority: 110 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 9775 sha1: bbeb098e76b9019fafbe1c7bd55e48604c324c89 sleep_before_teardown: 0 subset: 55/120000 suite: rados suite_branch: wip-rf-repl-hp suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bbeb098e76b9019fafbe1c7bd55e48604c324c89/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bbeb098e76b9019fafbe1c7bd55e48604c324c89 targets: smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK4CcJFpY/k+yeFg5uhqfolGWCEzEOC4S+Tv3otpRlwbuKIRwzhZwd2eOZ/bYw3QV/zDCwmDuj2AAKiVitqtaOo= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: f2de108beb0a6dde7f3e6c96039071828a1f6269 timestamp: 2023-11-30_18:48:32 tube: smithi user: rfriedma verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-01T06:09:02.708 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bbeb098e76b9019fafbe1c7bd55e48604c324c89/qa; will attempt to use it 2023-12-01T06:09:02.708 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bbeb098e76b9019fafbe1c7bd55e48604c324c89/qa/tasks 2023-12-01T06:09:02.708 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-01T06:09:02.710 INFO:teuthology.task.internal:Checking packages... 2023-12-01T06:09:02.734 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'bbeb098e76b9019fafbe1c7bd55e48604c324c89' 2023-12-01T06:09:02.734 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-01T06:09:02.734 INFO:teuthology.packaging:ref: None 2023-12-01T06:09:02.735 INFO:teuthology.packaging:tag: None 2023-12-01T06:09:02.735 INFO:teuthology.packaging:branch: wip-rf-repl-hp 2023-12-01T06:09:02.735 INFO:teuthology.packaging:sha1: bbeb098e76b9019fafbe1c7bd55e48604c324c89 2023-12-01T06:09:02.735 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-rf-repl-hp 2023-12-01T06:09:02.980 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7432-gbbeb098e-1jammy 2023-12-01T06:09:02.980 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-01T06:09:02.989 INFO:teuthology.task.internal:no buildpackages task found 2023-12-01T06:09:02.989 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-01T06:09:03.018 INFO:teuthology.task.internal:Saving configuration 2023-12-01T06:09:03.033 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-01T06:09:03.056 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-01T06:09:03.085 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rfriedma-2023-11-30_18:48:32-rados-wip-rf-repl-hp-distro-default-smithi/7472795', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-01 06:03:37.073203', 'locked_by': 'scheduled_rfriedma@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYRAqsLcJocCSrUlxwvzJkVsvN2t43q0Hb+lzLaimuN'} 2023-12-01T06:09:03.085 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-01T06:09:03.091 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2023-12-01T06:09:03.091 INFO:teuthology.run_tasks:Running task console_log... 2023-12-01T06:09:03.151 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f59162329d0>, signals=[15]) 2023-12-01T06:09:03.151 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-01T06:09:03.158 INFO:teuthology.task.internal:Opening connections... 2023-12-01T06:09:03.158 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2023-12-01T06:09:03.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T06:09:03.233 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-01T06:09:03.240 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2023-12-01T06:09:03.351 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2023-12-01T06:09:03.351 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2023-12-01T06:09:03.396 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-01T06:09:03.396 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Ubuntu" 2023-12-01T06:09:03.397 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="22.04" 2023-12-01T06:09:03.397 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-01T06:09:03.397 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_CODENAME=jammy 2023-12-01T06:09:03.397 INFO:teuthology.orchestra.run.smithi170.stdout:ID=ubuntu 2023-12-01T06:09:03.397 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE=debian 2023-12-01T06:09:03.397 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-01T06:09:03.397 INFO:teuthology.orchestra.run.smithi170.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-01T06:09:03.397 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-01T06:09:03.397 INFO:teuthology.orchestra.run.smithi170.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-01T06:09:03.397 INFO:teuthology.orchestra.run.smithi170.stdout:UBUNTU_CODENAME=jammy 2023-12-01T06:09:03.398 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2023-12-01T06:09:03.430 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-01T06:09:03.439 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-01T06:09:03.476 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-01T06:09:03.477 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2023-12-01T06:09:03.482 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-01T06:09:03.501 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-01T06:09:03.502 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2023-12-01T06:09:03.530 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-01T06:09:03.563 INFO:teuthology.run_tasks:Running task kernel... 2023-12-01T06:09:03.578 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-12-01T06:09:03.578 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-12-01T06:09:03.579 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-01T06:09:03.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T06:09:03.584 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2023-12-01T06:09:03.633 INFO:teuthology.orchestra.run.smithi170.stdout:5.15.0-56-generic 2023-12-01T06:09:03.633 INFO:teuthology.task.kernel:Running kernel on smithi170: 5.15.0-56-generic 2023-12-01T06:09:03.634 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get clean 2023-12-01T06:09:03.778 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2023-12-01T06:09:03.917 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-12-01T06:09:03.918 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-12-01T06:09:03.918 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2023-12-01T06:09:03.988 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-12-01T06:09:04.160 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,211 kB] 2023-12-01T06:09:04.236 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,002 kB] 2023-12-01T06:09:04.244 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [537 kB] 2023-12-01T06:09:04.248 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [253 kB] 2023-12-01T06:09:04.251 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2023-12-01T06:09:04.251 INFO:teuthology.orchestra.run.smithi170.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [32.8 kB] 2023-12-01T06:09:04.251 INFO:teuthology.orchestra.run.smithi170.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,185 kB] 2023-12-01T06:09:04.299 INFO:teuthology.orchestra.run.smithi170.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [192 kB] 2023-12-01T06:09:04.301 INFO:teuthology.orchestra.run.smithi170.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2023-12-01T06:09:04.301 INFO:teuthology.orchestra.run.smithi170.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,010 kB] 2023-12-01T06:09:04.312 INFO:teuthology.orchestra.run.smithi170.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [671 kB] 2023-12-01T06:09:04.318 INFO:teuthology.orchestra.run.smithi170.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [224 kB] 2023-12-01T06:09:04.321 INFO:teuthology.orchestra.run.smithi170.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2023-12-01T06:09:04.321 INFO:teuthology.orchestra.run.smithi170.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [3,888 B] 2023-12-01T06:09:04.322 INFO:teuthology.orchestra.run.smithi170.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] 2023-12-01T06:09:04.322 INFO:teuthology.orchestra.run.smithi170.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9,768 B] 2023-12-01T06:09:04.323 INFO:teuthology.orchestra.run.smithi170.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2023-12-01T06:09:04.332 INFO:teuthology.orchestra.run.smithi170.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [373 kB] 2023-12-01T06:09:04.334 INFO:teuthology.orchestra.run.smithi170.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [56.3 kB] 2023-12-01T06:09:04.335 INFO:teuthology.orchestra.run.smithi170.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [64.2 kB] 2023-12-01T06:09:04.335 INFO:teuthology.orchestra.run.smithi170.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2023-12-01T06:09:04.337 INFO:teuthology.orchestra.run.smithi170.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [193 kB] 2023-12-01T06:09:04.340 INFO:teuthology.orchestra.run.smithi170.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2023-12-01T06:09:04.341 INFO:teuthology.orchestra.run.smithi170.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,164 kB] 2023-12-01T06:09:04.351 INFO:teuthology.orchestra.run.smithi170.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2023-12-01T06:09:04.351 INFO:teuthology.orchestra.run.smithi170.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.8 kB] 2023-12-01T06:09:04.352 INFO:teuthology.orchestra.run.smithi170.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.8 kB] 2023-12-01T06:09:04.352 INFO:teuthology.orchestra.run.smithi170.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2023-12-01T06:09:04.352 INFO:teuthology.orchestra.run.smithi170.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2023-12-01T06:09:04.362 INFO:teuthology.orchestra.run.smithi170.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [32.4 kB] 2023-12-01T06:09:04.362 INFO:teuthology.orchestra.run.smithi170.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [189 kB] 2023-12-01T06:09:04.365 INFO:teuthology.orchestra.run.smithi170.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2023-12-01T06:09:04.366 INFO:teuthology.orchestra.run.smithi170.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [807 kB] 2023-12-01T06:09:04.377 INFO:teuthology.orchestra.run.smithi170.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [573 kB] 2023-12-01T06:09:04.382 INFO:teuthology.orchestra.run.smithi170.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [150 kB] 2023-12-01T06:09:04.383 INFO:teuthology.orchestra.run.smithi170.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2023-12-01T06:09:04.384 INFO:teuthology.orchestra.run.smithi170.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] 2023-12-01T06:09:04.384 INFO:teuthology.orchestra.run.smithi170.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,032 B] 2023-12-01T06:09:04.404 INFO:teuthology.orchestra.run.smithi170.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,060 B] 2023-12-01T06:09:04.404 INFO:teuthology.orchestra.run.smithi170.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2023-12-01T06:09:09.917 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 10.5 MB in 2s (6,875 kB/s) 2023-12-01T06:09:11.312 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2023-12-01T06:09:11.330 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-12-01T06:09:11.384 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2023-12-01T06:09:11.561 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2023-12-01T06:09:11.561 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2023-12-01T06:09:11.703 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2023-12-01T06:09:11.703 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-01T06:09:11.703 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-01T06:09:11.704 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-01T06:09:11.704 INFO:teuthology.orchestra.run.smithi170.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-12-01T06:09:11.704 INFO:teuthology.orchestra.run.smithi170.stdout: python2.7-minimal 2023-12-01T06:09:11.704 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-01T06:09:11.705 INFO:teuthology.orchestra.run.smithi170.stdout:The following additional packages will be installed: 2023-12-01T06:09:11.705 INFO:teuthology.orchestra.run.smithi170.stdout: linux-generic linux-headers-5.15.0-89 linux-headers-5.15.0-89-generic 2023-12-01T06:09:11.705 INFO:teuthology.orchestra.run.smithi170.stdout: linux-headers-generic linux-image-5.15.0-89-generic 2023-12-01T06:09:11.705 INFO:teuthology.orchestra.run.smithi170.stdout: linux-modules-5.15.0-89-generic linux-modules-extra-5.15.0-89-generic 2023-12-01T06:09:11.706 INFO:teuthology.orchestra.run.smithi170.stdout:Suggested packages: 2023-12-01T06:09:11.706 INFO:teuthology.orchestra.run.smithi170.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2023-12-01T06:09:11.706 INFO:teuthology.orchestra.run.smithi170.stdout:Recommended packages: 2023-12-01T06:09:11.706 INFO:teuthology.orchestra.run.smithi170.stdout: thermald 2023-12-01T06:09:11.755 INFO:teuthology.orchestra.run.smithi170.stdout:The following NEW packages will be installed: 2023-12-01T06:09:11.755 INFO:teuthology.orchestra.run.smithi170.stdout: linux-headers-5.15.0-89 linux-headers-5.15.0-89-generic 2023-12-01T06:09:11.756 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.15.0-89-generic linux-modules-5.15.0-89-generic 2023-12-01T06:09:11.756 INFO:teuthology.orchestra.run.smithi170.stdout: linux-modules-extra-5.15.0-89-generic 2023-12-01T06:09:11.757 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages will be upgraded: 2023-12-01T06:09:11.757 INFO:teuthology.orchestra.run.smithi170.stdout: linux-generic linux-headers-generic linux-image-generic 2023-12-01T06:09:11.935 INFO:teuthology.orchestra.run.smithi170.stdout:3 upgraded, 5 newly installed, 0 to remove and 274 not upgraded. 2023-12-01T06:09:11.935 INFO:teuthology.orchestra.run.smithi170.stdout:Need to get 113 MB of archives. 2023-12-01T06:09:11.935 INFO:teuthology.orchestra.run.smithi170.stdout:After this operation, 582 MB of additional disk space will be used. 2023-12-01T06:09:11.935 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-89-generic amd64 5.15.0-89.99 [22.4 MB] 2023-12-01T06:09:13.329 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-89-generic amd64 5.15.0-89.99 [11.5 MB] 2023-12-01T06:09:13.814 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-89-generic amd64 5.15.0-89.99 [63.9 MB] 2023-12-01T06:09:15.723 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.89.86 [1,698 B] 2023-12-01T06:09:15.756 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.89.86 [2,492 B] 2023-12-01T06:09:15.791 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-89 all 5.15.0-89.99 [12.3 MB] 2023-12-01T06:09:16.141 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-89-generic amd64 5.15.0-89.99 [2,885 kB] 2023-12-01T06:09:16.230 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.89.86 [2,336 B] 2023-12-01T06:09:16.629 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 113 MB in 4s (25.3 MB/s) 2023-12-01T06:09:16.790 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-modules-5.15.0-89-generic. 2023-12-01T06:09:18.795 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2023-12-01T06:09:18.799 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../0-linux-modules-5.15.0-89-generic_5.15.0-89.99_amd64.deb ... 2023-12-01T06:09:18.879 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-modules-5.15.0-89-generic (5.15.0-89.99) ... 2023-12-01T06:09:21.006 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-image-5.15.0-89-generic. 2023-12-01T06:09:21.027 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../1-linux-image-5.15.0-89-generic_5.15.0-89.99_amd64.deb ... 2023-12-01T06:09:21.105 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-image-5.15.0-89-generic (5.15.0-89.99) ... 2023-12-01T06:09:21.550 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-89-generic. 2023-12-01T06:09:21.570 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-89-generic_5.15.0-89.99_amd64.deb ... 2023-12-01T06:09:21.608 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-modules-extra-5.15.0-89-generic (5.15.0-89.99) ... 2023-12-01T06:09:28.302 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../3-linux-generic_5.15.0.89.86_amd64.deb ... 2023-12-01T06:09:28.410 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-generic (5.15.0.89.86) over (5.15.0.56.54) ... 2023-12-01T06:09:28.684 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.89.86_amd64.deb ... 2023-12-01T06:09:28.788 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-image-generic (5.15.0.89.86) over (5.15.0.56.54) ... 2023-12-01T06:09:29.083 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-headers-5.15.0-89. 2023-12-01T06:09:29.105 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../5-linux-headers-5.15.0-89_5.15.0-89.99_all.deb ... 2023-12-01T06:09:29.149 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-5.15.0-89 (5.15.0-89.99) ... 2023-12-01T06:09:35.003 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-headers-5.15.0-89-generic. 2023-12-01T06:09:35.023 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../6-linux-headers-5.15.0-89-generic_5.15.0-89.99_amd64.deb ... 2023-12-01T06:09:35.061 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-5.15.0-89-generic (5.15.0-89.99) ... 2023-12-01T06:09:37.647 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.89.86_amd64.deb ... 2023-12-01T06:09:37.773 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-generic (5.15.0.89.86) over (5.15.0.56.54) ... 2023-12-01T06:09:38.122 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-5.15.0-89 (5.15.0-89.99) ... 2023-12-01T06:09:38.244 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-5.15.0-89-generic (5.15.0-89.99) ... 2023-12-01T06:09:38.373 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-generic (5.15.0.89.86) ... 2023-12-01T06:09:38.500 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-image-5.15.0-89-generic (5.15.0-89.99) ... 2023-12-01T06:09:39.850 INFO:teuthology.orchestra.run.smithi170.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2023-12-01T06:09:39.850 INFO:teuthology.orchestra.run.smithi170.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2023-12-01T06:09:39.850 INFO:teuthology.orchestra.run.smithi170.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-89-generic 2023-12-01T06:09:39.850 INFO:teuthology.orchestra.run.smithi170.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-89-generic 2023-12-01T06:09:40.084 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-modules-extra-5.15.0-89-generic (5.15.0-89.99) ... 2023-12-01T06:09:41.519 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-image-generic (5.15.0.89.86) ... 2023-12-01T06:09:41.645 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-generic (5.15.0.89.86) ... 2023-12-01T06:09:41.771 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-modules-5.15.0-89-generic (5.15.0-89.99) ... 2023-12-01T06:09:43.173 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for linux-image-5.15.0-89-generic (5.15.0-89.99) ... 2023-12-01T06:09:43.236 INFO:teuthology.orchestra.run.smithi170.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-12-01T06:09:43.236 INFO:teuthology.orchestra.run.smithi170.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-89-generic 2023-12-01T06:09:57.615 INFO:teuthology.orchestra.run.smithi170.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-12-01T06:09:57.615 INFO:teuthology.orchestra.run.smithi170.stdout:Sourcing file `/etc/default/grub' 2023-12-01T06:09:57.627 INFO:teuthology.orchestra.run.smithi170.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-01T06:09:57.649 INFO:teuthology.orchestra.run.smithi170.stdout:Generating grub configuration file ... 2023-12-01T06:09:57.975 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.15.0-89-generic 2023-12-01T06:09:57.994 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.15.0-89-generic 2023-12-01T06:09:58.412 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-12-01T06:09:58.417 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-12-01T06:09:58.487 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-12-01T06:09:58.494 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-12-01T06:09:58.652 INFO:teuthology.orchestra.run.smithi170.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2023-12-01T06:09:58.652 INFO:teuthology.orchestra.run.smithi170.stdout:Systems on them will not be added to the GRUB boot configuration. 2023-12-01T06:09:58.652 INFO:teuthology.orchestra.run.smithi170.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-12-01T06:09:58.691 INFO:teuthology.orchestra.run.smithi170.stdout:done 2023-12-01T06:09:59.145 DEBUG:teuthology.orchestra.run.smithi170:> dpkg -s linux-image-generic 2023-12-01T06:09:59.211 INFO:teuthology.orchestra.run.smithi170.stdout:Package: linux-image-generic 2023-12-01T06:09:59.211 INFO:teuthology.orchestra.run.smithi170.stdout:Status: install ok installed 2023-12-01T06:09:59.211 INFO:teuthology.orchestra.run.smithi170.stdout:Priority: optional 2023-12-01T06:09:59.211 INFO:teuthology.orchestra.run.smithi170.stdout:Section: kernel 2023-12-01T06:09:59.212 INFO:teuthology.orchestra.run.smithi170.stdout:Installed-Size: 20 2023-12-01T06:09:59.212 INFO:teuthology.orchestra.run.smithi170.stdout:Maintainer: Ubuntu Kernel Team 2023-12-01T06:09:59.212 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: amd64 2023-12-01T06:09:59.212 INFO:teuthology.orchestra.run.smithi170.stdout:Source: linux-meta 2023-12-01T06:09:59.212 INFO:teuthology.orchestra.run.smithi170.stdout:Version: 5.15.0.89.86 2023-12-01T06:09:59.212 INFO:teuthology.orchestra.run.smithi170.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-89), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-12-01T06:09:59.212 INFO:teuthology.orchestra.run.smithi170.stdout:Depends: linux-image-5.15.0-89-generic, linux-modules-extra-5.15.0-89-generic, linux-firmware, intel-microcode, amd64-microcode 2023-12-01T06:09:59.212 INFO:teuthology.orchestra.run.smithi170.stdout:Recommends: thermald 2023-12-01T06:09:59.212 INFO:teuthology.orchestra.run.smithi170.stdout:Description: Generic Linux kernel image 2023-12-01T06:09:59.212 INFO:teuthology.orchestra.run.smithi170.stdout: This package will always depend on the latest generic kernel image 2023-12-01T06:09:59.212 INFO:teuthology.orchestra.run.smithi170.stdout: available. 2023-12-01T06:09:59.212 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-89-generic 2023-12-01T06:09:59.212 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-01T06:09:59.213 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-01T06:09:59.213 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-12-01T06:09:59.213 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi170.front.sepia.ceph.com, path=None, version=distro) 2023-12-01T06:09:59.213 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get clean 2023-12-01T06:09:59.379 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2023-12-01T06:09:59.514 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-12-01T06:09:59.628 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-12-01T06:09:59.710 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-12-01T06:09:59.806 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-12-01T06:10:00.868 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2023-12-01T06:10:00.885 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-12-01T06:10:00.945 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2023-12-01T06:10:01.139 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2023-12-01T06:10:01.139 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2023-12-01T06:10:01.306 INFO:teuthology.orchestra.run.smithi170.stdout:linux-image-generic is already the newest version (5.15.0.89.86). 2023-12-01T06:10:01.306 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2023-12-01T06:10:01.306 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-01T06:10:01.306 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-01T06:10:01.306 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-01T06:10:01.307 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-01T06:10:01.307 INFO:teuthology.orchestra.run.smithi170.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-01T06:10:01.307 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-01T06:10:01.346 INFO:teuthology.orchestra.run.smithi170.stdout:0 upgraded, 0 newly installed, 0 to remove and 274 not upgraded. 2023-12-01T06:10:01.348 DEBUG:teuthology.orchestra.run.smithi170:> dpkg -s linux-image-generic 2023-12-01T06:10:01.361 INFO:teuthology.orchestra.run.smithi170.stdout:Package: linux-image-generic 2023-12-01T06:10:01.361 INFO:teuthology.orchestra.run.smithi170.stdout:Status: install ok installed 2023-12-01T06:10:01.361 INFO:teuthology.orchestra.run.smithi170.stdout:Priority: optional 2023-12-01T06:10:01.361 INFO:teuthology.orchestra.run.smithi170.stdout:Section: kernel 2023-12-01T06:10:01.361 INFO:teuthology.orchestra.run.smithi170.stdout:Installed-Size: 20 2023-12-01T06:10:01.361 INFO:teuthology.orchestra.run.smithi170.stdout:Maintainer: Ubuntu Kernel Team 2023-12-01T06:10:01.361 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: amd64 2023-12-01T06:10:01.361 INFO:teuthology.orchestra.run.smithi170.stdout:Source: linux-meta 2023-12-01T06:10:01.361 INFO:teuthology.orchestra.run.smithi170.stdout:Version: 5.15.0.89.86 2023-12-01T06:10:01.361 INFO:teuthology.orchestra.run.smithi170.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-89), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-12-01T06:10:01.362 INFO:teuthology.orchestra.run.smithi170.stdout:Depends: linux-image-5.15.0-89-generic, linux-modules-extra-5.15.0-89-generic, linux-firmware, intel-microcode, amd64-microcode 2023-12-01T06:10:01.362 INFO:teuthology.orchestra.run.smithi170.stdout:Recommends: thermald 2023-12-01T06:10:01.362 INFO:teuthology.orchestra.run.smithi170.stdout:Description: Generic Linux kernel image 2023-12-01T06:10:01.362 INFO:teuthology.orchestra.run.smithi170.stdout: This package will always depend on the latest generic kernel image 2023-12-01T06:10:01.362 INFO:teuthology.orchestra.run.smithi170.stdout: available. 2023-12-01T06:10:01.362 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2023-12-01T06:10:01.409 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.d7ukgCqvvg 2023-12-01T06:10:01.409 DEBUG:teuthology.orchestra.run.smithi170:> sudo cp /boot/grub/grub.cfg /tmp/tmp.d7ukgCqvvg 2023-12-01T06:10:01.474 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.d7ukgCqvvg 2023-12-01T06:10:01.613 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.d7ukgCqvvg is 10KB 2023-12-01T06:10:01.626 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.d7ukgCqvvg 2023-12-01T06:10:01.631 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-12-01T06:10:01.685 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-12-01T06:10:01.685 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-12-01T06:10:01.685 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-12-01T06:10:01.785 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-89-generic 2023-12-01T06:10:01.785 DEBUG:teuthology.orchestra.run.smithi170:> sudo update-grub 2023-12-01T06:10:02.988 INFO:teuthology.orchestra.run.smithi170.stderr:Sourcing file `/etc/default/grub' 2023-12-01T06:10:02.990 INFO:teuthology.orchestra.run.smithi170.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-01T06:10:02.993 INFO:teuthology.orchestra.run.smithi170.stderr:Generating grub configuration file ... 2023-12-01T06:10:03.257 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.15.0-89-generic 2023-12-01T06:10:03.264 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.15.0-89-generic 2023-12-01T06:10:03.572 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-12-01T06:10:03.578 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-12-01T06:10:03.620 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-12-01T06:10:03.625 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-12-01T06:10:03.779 INFO:teuthology.orchestra.run.smithi170.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-12-01T06:10:03.779 INFO:teuthology.orchestra.run.smithi170.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-12-01T06:10:03.779 INFO:teuthology.orchestra.run.smithi170.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-12-01T06:10:03.804 INFO:teuthology.orchestra.run.smithi170.stderr:done 2023-12-01T06:10:03.813 DEBUG:teuthology.orchestra.run.smithi170:> sudo shutdown -r now 2023-12-01T06:10:33.846 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-12-01T06:10:33.846 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-12-01T06:10:33.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T06:10:52.386 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-12-01T06:11:01.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-12-01T06:11:01.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T06:11:04.450 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-12-01T06:11:16.462 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-12-01T06:11:16.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T06:11:34.882 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-12-01T06:11:49.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-12-01T06:11:49.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T06:11:50.463 DEBUG:teuthology.orchestra.run.smithi170:> true 2023-12-01T06:11:50.980 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-12-01T06:11:50.981 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-89-generic"... 2023-12-01T06:11:50.981 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2023-12-01T06:11:51.026 INFO:teuthology.orchestra.run.smithi170.stdout:5.15.0-89-generic 2023-12-01T06:11:51.026 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-89-generic vs 5.15.0-89-generic 2023-12-01T06:11:51.027 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-01T06:11:51.027 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-12-01T06:11:52.028 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-12-01T06:11:52.028 DEBUG:teuthology.orchestra.run.smithi170:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-01T06:11:52.106 INFO:teuthology.orchestra.run.smithi170.stdout:ttyS1 2023-12-01T06:11:52.132 DEBUG:teuthology.parallel:result is None 2023-12-01T06:11:52.132 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-01T06:11:52.140 INFO:teuthology.task.internal:Creating test directory... 2023-12-01T06:11:52.140 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-01T06:11:52.146 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-01T06:11:52.182 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-01T06:11:52.240 INFO:teuthology.task.internal:Creating archive directory... 2023-12-01T06:11:52.241 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-01T06:11:52.258 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-01T06:11:52.273 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-01T06:11:52.274 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-01T06:11:52.315 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:11:52.325 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:11:52.325 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-01T06:11:52.333 INFO:teuthology.task.internal:Configuring sudo... 2023-12-01T06:11:52.333 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-01T06:11:52.378 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-01T06:11:52.387 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-01T06:11:52.387 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-01T06:11:52.427 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-01T06:11:52.475 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-01T06:11:52.523 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-12-01T06:11:52.523 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-01T06:11:52.594 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2023-12-01T06:11:52.701 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-01T06:11:52.709 INFO:teuthology.task.internal:Starting timer... 2023-12-01T06:11:52.709 INFO:teuthology.run_tasks:Running task pcp... 2023-12-01T06:11:52.743 INFO:teuthology.run_tasks:Running task selinux... 2023-12-01T06:11:52.776 DEBUG:teuthology.task.selinux:Excluding smithi170: OS 'ubuntu' does not support SELinux 2023-12-01T06:11:52.777 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-01T06:11:52.777 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-12-01T06:11:52.777 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-01T06:11:52.777 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-01T06:11:52.810 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-12-01T06:11:52.810 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-01T06:11:52.828 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-01T06:11:52.829 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-01T06:19:48.324 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2023-12-01T06:19:48.325 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-12-01T06:19:48.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T06:19:48.402 DEBUG:teuthology.orchestra.run.smithi170:> true 2023-12-01T06:19:48.527 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-12-01T06:19:48.527 INFO:teuthology.run_tasks:Running task clock... 2023-12-01T06:19:48.545 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-01T06:19:48.546 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-01T06:19:48.546 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-01T06:19:48.595 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-12-01T06:19:48.596 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: Command line: ntpd -gq 2023-12-01T06:19:48.596 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: ---------------------------------------------------- 2023-12-01T06:19:48.596 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: ntp-4 is maintained by Network Time Foundation, 2023-12-01T06:19:48.596 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-12-01T06:19:48.596 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: corporation. Support and training for ntp-4 are 2023-12-01T06:19:48.597 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: available at https://www.nwtime.org/support 2023-12-01T06:19:48.597 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: ---------------------------------------------------- 2023-12-01T06:19:48.597 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: proto: precision = 0.062 usec (-24) 2023-12-01T06:19:48.599 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: basedate set to 2022-02-04 2023-12-01T06:19:48.599 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: gps base set to 2022-02-06 (week 2196) 2023-12-01T06:19:48.599 INFO:teuthology.orchestra.run.smithi170.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-01T06:19:48.600 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-01T06:19:48.600 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: restrict ::: KOD does nothing without LIMITED. 2023-12-01T06:19:48.600 INFO:teuthology.orchestra.run.smithi170.stderr:restrict ::: KOD does nothing without LIMITED. 2023-12-01T06:19:48.601 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: Listen and drop on 0 v6wildcard [::]:123 2023-12-01T06:19:48.601 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-12-01T06:19:48.601 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: Listen normally on 2 lo 127.0.0.1:123 2023-12-01T06:19:48.602 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: Listen normally on 3 enp3s0f1 172.21.15.170:123 2023-12-01T06:19:48.602 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: Listen normally on 4 lo [::1]:123 2023-12-01T06:19:48.602 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7255%5]:123 2023-12-01T06:19:48.602 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:48 ntpd[27985]: Listening on routing socket on fd #22 for interface updates 2023-12-01T06:19:50.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:50 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:50.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:50 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:50.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:50 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:50.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:50 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:50.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:50 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:50.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:50 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:50.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:50 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-01T06:19:50.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:50 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231201: Permission denied 2023-12-01T06:19:52.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:52 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:52.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:52 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:52.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:52 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:52.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:52 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:52.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:52 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:52.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:52 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:52.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:52 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-01T06:19:52.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:52 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231201: Permission denied 2023-12-01T06:19:52.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:52 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:52.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:52 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:52.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:52 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-01T06:19:52.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:52 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231201: Permission denied 2023-12-01T06:19:53.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:53 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:53.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:53 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:53.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:53 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:53.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:53 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:53.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:53 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-01T06:19:53.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:53 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231201: Permission denied 2023-12-01T06:19:54.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:54 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:54.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:54 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:54.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:54 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:54.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:54 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:54.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:54 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:54.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:54 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:54.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:54 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-01T06:19:54.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:54 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231201: Permission denied 2023-12-01T06:19:54.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:54 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:54.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:54 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:54.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:54 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-01T06:19:54.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:54 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231201: Permission denied 2023-12-01T06:19:55.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:55 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:55.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:55 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:55.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:55 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:55.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:55 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:55.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:55 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-01T06:19:55.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:55 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231201: Permission denied 2023-12-01T06:19:56.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:56 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:56.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:56 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:56.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:56 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:56.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:56 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:56.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:56 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-01T06:19:56.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:56 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231201: Permission denied 2023-12-01T06:19:56.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:56 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-01T06:19:56.596 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:56 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231201: Permission denied 2023-12-01T06:19:56.597 INFO:teuthology.orchestra.run.smithi170.stdout: 1 Dec 06:19:56 ntpd[27985]: ntpd: time slew -0.000495 s 2023-12-01T06:19:56.597 INFO:teuthology.orchestra.run.smithi170.stdout:ntpd: time slew -0.000495s 2023-12-01T06:19:56.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:56 ntpd[27985]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-12-01T06:19:56.597 INFO:teuthology.orchestra.run.smithi170.stderr: 1 Dec 06:19:56 ntpd[27985]: can't open /var/log/ntpstats/loopstats.20231201: Permission denied 2023-12-01T06:19:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: remote refid st t when poll reach delay offset jitter 2023-12-01T06:19:56.687 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================== 2023-12-01T06:19:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-01T06:19:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-01T06:19:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-01T06:19:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-01T06:19:56.688 INFO:teuthology.run_tasks:Running task install... 2023-12-01T06:19:56.698 DEBUG:teuthology.task.install:project ceph 2023-12-01T06:19:56.698 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bbeb098e76b9019fafbe1c7bd55e48604c324c89'}} 2023-12-01T06:19:56.698 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'bbeb098e76b9019fafbe1c7bd55e48604c324c89'} 2023-12-01T06:19:56.698 INFO:teuthology.task.install:Using flavor: default 2023-12-01T06:19:56.704 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-01T06:19:56.704 INFO:teuthology.task.install:extra packages: [] 2023-12-01T06:19:56.705 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-key list | grep Ceph 2023-12-01T06:19:56.787 INFO:teuthology.orchestra.run.smithi170.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-12-01T06:19:56.813 INFO:teuthology.orchestra.run.smithi170.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-12-01T06:19:56.813 INFO:teuthology.orchestra.run.smithi170.stdout:uid [ unknown] Ceph.com (release key) 2023-12-01T06:19:56.814 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2023-12-01T06:19:56.814 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=bbeb098e76b9019fafbe1c7bd55e48604c324c89 2023-12-01T06:19:56.960 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default/ 2023-12-01T06:19:56.960 INFO:teuthology.task.install.deb:Package version is 18.0.0-7432-gbbeb098e-1jammy 2023-12-01T06:19:57.276 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-12-01T06:19:57.277 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-12-01T06:19:57.291 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2023-12-01T06:19:57.455 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-12-01T06:19:57.575 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-12-01T06:19:57.657 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-12-01T06:19:57.739 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-12-01T06:19:57.783 INFO:teuthology.orchestra.run.smithi170.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy InRelease 2023-12-01T06:19:57.851 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-12-01T06:19:57.939 INFO:teuthology.orchestra.run.smithi170.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy Release.gpg 2023-12-01T06:19:58.013 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-12-01T06:19:58.095 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,415 B] 2023-12-01T06:19:58.176 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 28.5 kB in 1s (40.7 kB/s) 2023-12-01T06:19:59.070 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2023-12-01T06:19:59.086 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-7432-gbbeb098e-1jammy cephadm=18.0.0-7432-gbbeb098e-1jammy ceph-mds=18.0.0-7432-gbbeb098e-1jammy ceph-mgr=18.0.0-7432-gbbeb098e-1jammy ceph-common=18.0.0-7432-gbbeb098e-1jammy ceph-fuse=18.0.0-7432-gbbeb098e-1jammy ceph-test=18.0.0-7432-gbbeb098e-1jammy ceph-volume=18.0.0-7432-gbbeb098e-1jammy radosgw=18.0.0-7432-gbbeb098e-1jammy python3-rados=18.0.0-7432-gbbeb098e-1jammy python3-rgw=18.0.0-7432-gbbeb098e-1jammy python3-cephfs=18.0.0-7432-gbbeb098e-1jammy python3-rbd=18.0.0-7432-gbbeb098e-1jammy libcephfs2=18.0.0-7432-gbbeb098e-1jammy libcephfs-dev=18.0.0-7432-gbbeb098e-1jammy librados2=18.0.0-7432-gbbeb098e-1jammy librbd1=18.0.0-7432-gbbeb098e-1jammy rbd-fuse=18.0.0-7432-gbbeb098e-1jammy 2023-12-01T06:19:59.147 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2023-12-01T06:19:59.334 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2023-12-01T06:19:59.334 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2023-12-01T06:19:59.507 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2023-12-01T06:19:59.507 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-01T06:19:59.507 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-01T06:19:59.507 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-01T06:19:59.508 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-01T06:19:59.508 INFO:teuthology.orchestra.run.smithi170.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-01T06:19:59.508 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-01T06:19:59.508 INFO:teuthology.orchestra.run.smithi170.stdout:The following additional packages will be installed: 2023-12-01T06:19:59.508 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-12-01T06:19:59.509 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-12-01T06:19:59.509 INFO:teuthology.orchestra.run.smithi170.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liboath0 libonig5 libpcre2-16-0 2023-12-01T06:19:59.509 INFO:teuthology.orchestra.run.smithi170.stdout: libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 librdkafka1 librgw2 2023-12-01T06:19:59.510 INFO:teuthology.orchestra.run.smithi170.stdout: libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli python-asyncssh-doc 2023-12-01T06:19:59.510 INFO:teuthology.orchestra.run.smithi170.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2023-12-01T06:19:59.510 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2023-12-01T06:19:59.510 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common python3-certifi python3-cheroot python3-cherrypy3 2023-12-01T06:19:59.510 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2023-12-01T06:19:59.510 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-12-01T06:19:59.510 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-12-01T06:19:59.510 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-12-01T06:19:59.510 INFO:teuthology.orchestra.run.smithi170.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-12-01T06:19:59.510 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-12-01T06:19:59.510 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-12-01T06:19:59.510 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytest python3-repoze.lru python3-requests python3-requests-oauthlib 2023-12-01T06:19:59.510 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2023-12-01T06:19:59.511 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2023-12-01T06:19:59.511 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2023-12-01T06:19:59.511 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2023-12-01T06:19:59.511 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2023-12-01T06:19:59.511 INFO:teuthology.orchestra.run.smithi170.stdout: socat xmlstarlet 2023-12-01T06:19:59.511 INFO:teuthology.orchestra.run.smithi170.stdout:Suggested packages: 2023-12-01T06:19:59.511 INFO:teuthology.orchestra.run.smithi170.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-12-01T06:19:59.511 INFO:teuthology.orchestra.run.smithi170.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-12-01T06:19:59.512 INFO:teuthology.orchestra.run.smithi170.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-12-01T06:19:59.512 INFO:teuthology.orchestra.run.smithi170.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-12-01T06:19:59.512 INFO:teuthology.orchestra.run.smithi170.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-12-01T06:19:59.512 INFO:teuthology.orchestra.run.smithi170.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-12-01T06:19:59.512 INFO:teuthology.orchestra.run.smithi170.stdout: python3-watchdog 2023-12-01T06:19:59.512 INFO:teuthology.orchestra.run.smithi170.stdout:Recommended packages: 2023-12-01T06:19:59.512 INFO:teuthology.orchestra.run.smithi170.stdout: btrfs-tools 2023-12-01T06:19:59.608 INFO:teuthology.orchestra.run.smithi170.stdout:The following NEW packages will be installed: 2023-12-01T06:19:59.608 INFO:teuthology.orchestra.run.smithi170.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-12-01T06:19:59.609 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-12-01T06:19:59.609 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2023-12-01T06:19:59.609 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 liboath0 2023-12-01T06:19:59.609 INFO:teuthology.orchestra.run.smithi170.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-12-01T06:19:59.610 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-12-01T06:19:59.610 INFO:teuthology.orchestra.run.smithi170.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-12-01T06:19:59.610 INFO:teuthology.orchestra.run.smithi170.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-12-01T06:19:59.610 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-12-01T06:19:59.610 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2023-12-01T06:19:59.610 INFO:teuthology.orchestra.run.smithi170.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-12-01T06:19:59.610 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-12-01T06:19:59.610 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-12-01T06:19:59.610 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-12-01T06:19:59.610 INFO:teuthology.orchestra.run.smithi170.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-12-01T06:19:59.610 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-12-01T06:19:59.611 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-12-01T06:19:59.611 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2023-12-01T06:19:59.611 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-12-01T06:19:59.611 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-12-01T06:19:59.611 INFO:teuthology.orchestra.run.smithi170.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-12-01T06:19:59.611 INFO:teuthology.orchestra.run.smithi170.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-12-01T06:19:59.611 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-12-01T06:19:59.611 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-12-01T06:19:59.611 INFO:teuthology.orchestra.run.smithi170.stdout: socat xmlstarlet 2023-12-01T06:19:59.673 INFO:teuthology.orchestra.run.smithi170.stdout:0 upgraded, 108 newly installed, 0 to remove and 262 not upgraded. 2023-12-01T06:19:59.673 INFO:teuthology.orchestra.run.smithi170.stdout:Need to get 185 MB of archives. 2023-12-01T06:19:59.673 INFO:teuthology.orchestra.run.smithi170.stdout:After this operation, 828 MB of additional disk space will be used. 2023-12-01T06:19:59.673 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-12-01T06:19:59.775 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-12-01T06:19:59.778 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-12-01T06:19:59.797 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-12-01T06:19:59.890 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-12-01T06:19:59.896 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-12-01T06:19:59.911 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-12-01T06:19:59.916 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-12-01T06:19:59.916 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-12-01T06:19:59.917 INFO:teuthology.orchestra.run.smithi170.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-12-01T06:19:59.930 INFO:teuthology.orchestra.run.smithi170.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-12-01T06:19:59.931 INFO:teuthology.orchestra.run.smithi170.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-12-01T06:19:59.931 INFO:teuthology.orchestra.run.smithi170.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-12-01T06:19:59.933 INFO:teuthology.orchestra.run.smithi170.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-12-01T06:19:59.934 INFO:teuthology.orchestra.run.smithi170.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-12-01T06:19:59.935 INFO:teuthology.orchestra.run.smithi170.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-12-01T06:19:59.939 INFO:teuthology.orchestra.run.smithi170.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-7432-gbbeb098e-1jammy [3,468 kB] 2023-12-01T06:19:59.954 INFO:teuthology.orchestra.run.smithi170.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-12-01T06:19:59.955 INFO:teuthology.orchestra.run.smithi170.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-12-01T06:19:59.955 INFO:teuthology.orchestra.run.smithi170.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-12-01T06:19:59.956 INFO:teuthology.orchestra.run.smithi170.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-12-01T06:19:59.960 INFO:teuthology.orchestra.run.smithi170.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-12-01T06:19:59.961 INFO:teuthology.orchestra.run.smithi170.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-12-01T06:19:59.961 INFO:teuthology.orchestra.run.smithi170.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-12-01T06:19:59.962 INFO:teuthology.orchestra.run.smithi170.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-12-01T06:19:59.974 INFO:teuthology.orchestra.run.smithi170.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-12-01T06:19:59.974 INFO:teuthology.orchestra.run.smithi170.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-12-01T06:19:59.994 INFO:teuthology.orchestra.run.smithi170.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-12-01T06:19:59.995 INFO:teuthology.orchestra.run.smithi170.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-12-01T06:19:59.996 INFO:teuthology.orchestra.run.smithi170.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-12-01T06:19:59.997 INFO:teuthology.orchestra.run.smithi170.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-12-01T06:20:00.039 INFO:teuthology.orchestra.run.smithi170.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-12-01T06:20:00.039 INFO:teuthology.orchestra.run.smithi170.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-12-01T06:20:00.040 INFO:teuthology.orchestra.run.smithi170.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-12-01T06:20:00.041 INFO:teuthology.orchestra.run.smithi170.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-12-01T06:20:00.044 INFO:teuthology.orchestra.run.smithi170.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-12-01T06:20:00.048 INFO:teuthology.orchestra.run.smithi170.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-12-01T06:20:00.049 INFO:teuthology.orchestra.run.smithi170.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-12-01T06:20:00.052 INFO:teuthology.orchestra.run.smithi170.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-12-01T06:20:00.053 INFO:teuthology.orchestra.run.smithi170.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-12-01T06:20:00.060 INFO:teuthology.orchestra.run.smithi170.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-12-01T06:20:00.078 INFO:teuthology.orchestra.run.smithi170.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-12-01T06:20:00.123 INFO:teuthology.orchestra.run.smithi170.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-12-01T06:20:00.205 INFO:teuthology.orchestra.run.smithi170.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-12-01T06:20:00.206 INFO:teuthology.orchestra.run.smithi170.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-12-01T06:20:00.208 INFO:teuthology.orchestra.run.smithi170.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-12-01T06:20:00.208 INFO:teuthology.orchestra.run.smithi170.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-12-01T06:20:00.209 INFO:teuthology.orchestra.run.smithi170.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-12-01T06:20:00.210 INFO:teuthology.orchestra.run.smithi170.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-12-01T06:20:00.228 INFO:teuthology.orchestra.run.smithi170.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-12-01T06:20:00.230 INFO:teuthology.orchestra.run.smithi170.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-12-01T06:20:00.231 INFO:teuthology.orchestra.run.smithi170.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-12-01T06:20:00.258 INFO:teuthology.orchestra.run.smithi170.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-12-01T06:20:00.258 INFO:teuthology.orchestra.run.smithi170.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-12-01T06:20:00.258 INFO:teuthology.orchestra.run.smithi170.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-12-01T06:20:00.259 INFO:teuthology.orchestra.run.smithi170.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-12-01T06:20:00.259 INFO:teuthology.orchestra.run.smithi170.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-12-01T06:20:00.259 INFO:teuthology.orchestra.run.smithi170.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-12-01T06:20:00.260 INFO:teuthology.orchestra.run.smithi170.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-12-01T06:20:00.261 INFO:teuthology.orchestra.run.smithi170.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-12-01T06:20:00.278 INFO:teuthology.orchestra.run.smithi170.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-12-01T06:20:00.286 INFO:teuthology.orchestra.run.smithi170.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-12-01T06:20:00.287 INFO:teuthology.orchestra.run.smithi170.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-12-01T06:20:00.288 INFO:teuthology.orchestra.run.smithi170.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-12-01T06:20:00.293 INFO:teuthology.orchestra.run.smithi170.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-12-01T06:20:00.323 INFO:teuthology.orchestra.run.smithi170.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-12-01T06:20:00.333 INFO:teuthology.orchestra.run.smithi170.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-12-01T06:20:00.338 INFO:teuthology.orchestra.run.smithi170.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-12-01T06:20:00.338 INFO:teuthology.orchestra.run.smithi170.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-12-01T06:20:00.339 INFO:teuthology.orchestra.run.smithi170.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-12-01T06:20:00.339 INFO:teuthology.orchestra.run.smithi170.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-12-01T06:20:00.342 INFO:teuthology.orchestra.run.smithi170.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-12-01T06:20:00.343 INFO:teuthology.orchestra.run.smithi170.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-12-01T06:20:00.354 INFO:teuthology.orchestra.run.smithi170.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-12-01T06:20:00.355 INFO:teuthology.orchestra.run.smithi170.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-12-01T06:20:00.369 INFO:teuthology.orchestra.run.smithi170.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-12-01T06:20:00.372 INFO:teuthology.orchestra.run.smithi170.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-12-01T06:20:00.373 INFO:teuthology.orchestra.run.smithi170.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-12-01T06:20:01.577 INFO:teuthology.orchestra.run.smithi170.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-7432-gbbeb098e-1jammy [3,111 kB] 2023-12-01T06:20:02.110 INFO:teuthology.orchestra.run.smithi170.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-7432-gbbeb098e-1jammy [691 kB] 2023-12-01T06:20:02.227 INFO:teuthology.orchestra.run.smithi170.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-7432-gbbeb098e-1jammy [356 kB] 2023-12-01T06:20:02.281 INFO:teuthology.orchestra.run.smithi170.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-7432-gbbeb098e-1jammy [32.8 kB] 2023-12-01T06:20:02.285 INFO:teuthology.orchestra.run.smithi170.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-7432-gbbeb098e-1jammy [180 kB] 2023-12-01T06:20:02.309 INFO:teuthology.orchestra.run.smithi170.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-7432-gbbeb098e-1jammy [63.5 kB] 2023-12-01T06:20:02.315 INFO:teuthology.orchestra.run.smithi170.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-7432-gbbeb098e-1jammy [330 kB] 2023-12-01T06:20:02.367 INFO:teuthology.orchestra.run.smithi170.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-7432-gbbeb098e-1jammy [6,229 kB] 2023-12-01T06:20:03.025 INFO:teuthology.orchestra.run.smithi170.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-7432-gbbeb098e-1jammy [112 kB] 2023-12-01T06:20:03.030 INFO:teuthology.orchestra.run.smithi170.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-7432-gbbeb098e-1jammy [439 kB] 2023-12-01T06:20:03.056 INFO:teuthology.orchestra.run.smithi170.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-7432-gbbeb098e-1jammy [22.6 MB] 2023-12-01T06:20:04.513 INFO:teuthology.orchestra.run.smithi170.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-7432-gbbeb098e-1jammy [6,899 kB] 2023-12-01T06:20:04.665 INFO:teuthology.orchestra.run.smithi170.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-7432-gbbeb098e-1jammy [238 kB] 2023-12-01T06:20:04.669 INFO:teuthology.orchestra.run.smithi170.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-7432-gbbeb098e-1jammy [124 kB] 2023-12-01T06:20:04.675 INFO:teuthology.orchestra.run.smithi170.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-7432-gbbeb098e-1jammy [1,496 kB] 2023-12-01T06:20:04.746 INFO:teuthology.orchestra.run.smithi170.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-7432-gbbeb098e-1jammy [9,955 kB] 2023-12-01T06:20:05.214 INFO:teuthology.orchestra.run.smithi170.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-7432-gbbeb098e-1jammy [31.9 MB] 2023-12-01T06:20:06.919 INFO:teuthology.orchestra.run.smithi170.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-7432-gbbeb098e-1jammy [14.0 kB] 2023-12-01T06:20:06.990 INFO:teuthology.orchestra.run.smithi170.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-7432-gbbeb098e-1jammy [839 kB] 2023-12-01T06:20:07.006 INFO:teuthology.orchestra.run.smithi170.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-7432-gbbeb098e-1jammy [2,144 kB] 2023-12-01T06:20:07.044 INFO:teuthology.orchestra.run.smithi170.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-7432-gbbeb098e-1jammy [802 kB] 2023-12-01T06:20:07.055 INFO:teuthology.orchestra.run.smithi170.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-7432-gbbeb098e-1jammy [139 kB] 2023-12-01T06:20:07.057 INFO:teuthology.orchestra.run.smithi170.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-7432-gbbeb098e-1jammy [1,900 kB] 2023-12-01T06:20:07.087 INFO:teuthology.orchestra.run.smithi170.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-7432-gbbeb098e-1jammy [8,625 kB] 2023-12-01T06:20:07.495 INFO:teuthology.orchestra.run.smithi170.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-7432-gbbeb098e-1jammy [14.2 kB] 2023-12-01T06:20:07.496 INFO:teuthology.orchestra.run.smithi170.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-7432-gbbeb098e-1jammy [48.6 MB] 2023-12-01T06:20:10.268 INFO:teuthology.orchestra.run.smithi170.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-7432-gbbeb098e-1jammy [121 kB] 2023-12-01T06:20:10.270 INFO:teuthology.orchestra.run.smithi170.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-7432-gbbeb098e-1jammy [38.3 kB] 2023-12-01T06:20:10.271 INFO:teuthology.orchestra.run.smithi170.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-7432-gbbeb098e-1jammy [12.3 MB] 2023-12-01T06:20:10.796 INFO:teuthology.orchestra.run.smithi170.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-rf-repl-hp/bbeb098e76b9019fafbe1c7bd55e48604c324c89/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-7432-gbbeb098e-1jammy [91.4 kB] 2023-12-01T06:20:11.339 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 185 MB in 11s (16.6 MB/s) 2023-12-01T06:20:11.450 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-12-01T06:20:11.538 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165983 files and directories currently installed.) 2023-12-01T06:20:11.544 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-12-01T06:20:11.624 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-12-01T06:20:12.189 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-12-01T06:20:12.215 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-12-01T06:20:12.319 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-12-01T06:20:12.656 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-12-01T06:20:12.682 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-12-01T06:20:12.722 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-12-01T06:20:13.125 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-12-01T06:20:13.152 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-01T06:20:13.192 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-01T06:20:13.579 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-12-01T06:20:13.605 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-01T06:20:13.645 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-01T06:20:14.007 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-12-01T06:20:14.033 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-01T06:20:14.073 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-01T06:20:14.426 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-12-01T06:20:14.453 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-12-01T06:20:14.485 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-12-01T06:20:14.779 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librados2. 2023-12-01T06:20:14.806 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../007-librados2_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:14.846 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librados2 (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:15.290 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librbd1. 2023-12-01T06:20:15.317 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../008-librbd1_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:15.357 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librbd1 (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:15.785 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libcephfs2. 2023-12-01T06:20:15.811 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../009-libcephfs2_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:15.851 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libcephfs2 (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:16.103 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rados. 2023-12-01T06:20:16.119 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../010-python3-rados_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:16.153 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rados (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:16.465 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-12-01T06:20:16.491 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../011-python3-ceph-argparse_18.0.0-7432-gbbeb098e-1jammy_all.deb ... 2023-12-01T06:20:16.531 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-ceph-argparse (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:16.809 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cephfs. 2023-12-01T06:20:16.835 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../012-python3-cephfs_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:16.875 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cephfs (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:17.203 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-ceph-common. 2023-12-01T06:20:17.230 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../013-python3-ceph-common_18.0.0-7432-gbbeb098e-1jammy_all.deb ... 2023-12-01T06:20:17.270 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-ceph-common (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:17.765 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-wcwidth. 2023-12-01T06:20:17.792 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-12-01T06:20:17.832 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-12-01T06:20:18.151 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-prettytable. 2023-12-01T06:20:18.177 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2023-12-01T06:20:18.218 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-12-01T06:20:18.445 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rbd. 2023-12-01T06:20:18.472 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../016-python3-rbd_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:18.503 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rbd (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:18.840 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-12-01T06:20:18.866 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-12-01T06:20:18.906 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-12-01T06:20:19.217 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librgw2. 2023-12-01T06:20:19.244 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../018-librgw2_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:19.284 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librgw2 (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:19.828 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rgw. 2023-12-01T06:20:19.854 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../019-python3-rgw_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:19.903 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rgw (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:20.173 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liboath0:amd64. 2023-12-01T06:20:20.188 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../020-liboath0_2.6.7-3build1_amd64.deb ... 2023-12-01T06:20:20.231 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-12-01T06:20:20.534 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libradosstriper1. 2023-12-01T06:20:20.560 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../021-libradosstriper1_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:20.600 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libradosstriper1 (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:21.012 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-common. 2023-12-01T06:20:21.038 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../022-ceph-common_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:21.078 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-common (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:22.401 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-base. 2023-12-01T06:20:22.428 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../023-ceph-base_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:22.485 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-base (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:23.338 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-12-01T06:20:23.365 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../024-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-12-01T06:20:23.405 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-12-01T06:20:23.666 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cheroot. 2023-12-01T06:20:23.693 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../025-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-12-01T06:20:23.724 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-12-01T06:20:24.019 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-12-01T06:20:24.046 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../026-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-12-01T06:20:24.085 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-12-01T06:20:24.397 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.text. 2023-12-01T06:20:24.423 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../027-python3-jaraco.text_3.6.0-2_all.deb ... 2023-12-01T06:20:24.463 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-12-01T06:20:24.749 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-12-01T06:20:24.776 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../028-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-12-01T06:20:24.816 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-12-01T06:20:25.086 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-tempora. 2023-12-01T06:20:25.112 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../029-python3-tempora_4.1.2-1_all.deb ... 2023-12-01T06:20:25.144 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-12-01T06:20:25.480 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-portend. 2023-12-01T06:20:25.507 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../030-python3-portend_3.0.0-1_all.deb ... 2023-12-01T06:20:25.547 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-12-01T06:20:25.849 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-12-01T06:20:25.876 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../031-python3-zc.lockfile_2.0-1_all.deb ... 2023-12-01T06:20:25.916 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-12-01T06:20:26.194 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cherrypy3. 2023-12-01T06:20:26.220 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../032-python3-cherrypy3_18.6.1-4_all.deb ... 2023-12-01T06:20:26.260 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-12-01T06:20:26.606 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-natsort. 2023-12-01T06:20:26.632 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../033-python3-natsort_8.0.2-1_all.deb ... 2023-12-01T06:20:26.672 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-12-01T06:20:26.984 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-logutils. 2023-12-01T06:20:27.010 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../034-python3-logutils_0.3.3-8_all.deb ... 2023-12-01T06:20:27.050 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-12-01T06:20:27.320 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-markupsafe. 2023-12-01T06:20:27.347 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../035-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-12-01T06:20:27.378 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-12-01T06:20:27.656 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-mako. 2023-12-01T06:20:27.683 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../036-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-12-01T06:20:27.722 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-12-01T06:20:28.076 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-simplegeneric. 2023-12-01T06:20:28.102 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../037-python3-simplegeneric_0.8.1-3_all.deb ... 2023-12-01T06:20:28.143 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-12-01T06:20:28.603 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-singledispatch. 2023-12-01T06:20:28.630 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../038-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-12-01T06:20:28.670 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-12-01T06:20:28.940 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-webob. 2023-12-01T06:20:28.966 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../039-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-12-01T06:20:28.998 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-12-01T06:20:29.300 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-waitress. 2023-12-01T06:20:29.327 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../040-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-12-01T06:20:29.362 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-12-01T06:20:29.678 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-tempita. 2023-12-01T06:20:29.705 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-12-01T06:20:29.745 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-12-01T06:20:30.040 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-paste. 2023-12-01T06:20:30.066 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../042-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-12-01T06:20:30.106 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-12-01T06:20:30.409 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-12-01T06:20:30.424 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-12-01T06:20:30.467 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-12-01T06:20:30.770 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pastedeploy. 2023-12-01T06:20:30.797 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../044-python3-pastedeploy_2.1.1-1_all.deb ... 2023-12-01T06:20:30.837 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-12-01T06:20:31.173 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-webtest. 2023-12-01T06:20:31.200 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../045-python3-webtest_2.0.35-1_all.deb ... 2023-12-01T06:20:31.240 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-12-01T06:20:31.534 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pecan. 2023-12-01T06:20:31.560 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../046-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-12-01T06:20:31.592 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-12-01T06:20:31.931 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-certifi. 2023-12-01T06:20:31.957 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../047-python3-certifi_2020.6.20-1_all.deb ... 2023-12-01T06:20:31.989 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-12-01T06:20:32.309 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-idna. 2023-12-01T06:20:32.335 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../048-python3-idna_3.3-1_all.deb ... 2023-12-01T06:20:32.375 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-idna (3.3-1) ... 2023-12-01T06:20:32.787 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-urllib3. 2023-12-01T06:20:32.813 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../049-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-12-01T06:20:33.162 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-12-01T06:20:33.732 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-requests. 2023-12-01T06:20:33.759 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../050-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-12-01T06:20:33.798 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-12-01T06:20:34.110 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-werkzeug. 2023-12-01T06:20:34.136 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../051-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-12-01T06:20:34.176 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-12-01T06:20:34.437 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-12-01T06:20:34.464 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../052-ceph-mgr-modules-core_18.0.0-7432-gbbeb098e-1jammy_all.deb ... 2023-12-01T06:20:34.504 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-modules-core (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:34.884 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-12-01T06:20:34.910 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../053-libsqlite3-mod-ceph_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:34.950 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:35.228 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-openssl. 2023-12-01T06:20:35.254 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../054-python3-openssl_21.0.0-1_all.deb ... 2023-12-01T06:20:35.286 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-12-01T06:20:35.606 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr. 2023-12-01T06:20:35.632 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../055-ceph-mgr_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:35.673 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:36.067 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mon. 2023-12-01T06:20:36.094 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../056-ceph-mon_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:36.134 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mon (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:36.988 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-osd. 2023-12-01T06:20:37.015 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../057-ceph-osd_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:37.045 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-osd (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:39.049 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph. 2023-12-01T06:20:39.076 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../058-ceph_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:39.115 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:39.460 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-fuse. 2023-12-01T06:20:39.487 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../059-ceph-fuse_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:39.526 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-fuse (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:39.854 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mds. 2023-12-01T06:20:39.881 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../060-ceph-mds_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:39.913 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mds (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:40.299 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package cephadm. 2023-12-01T06:20:40.327 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../061-cephadm_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:40.366 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking cephadm (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:40.677 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-asyncssh. 2023-12-01T06:20:40.704 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../062-python3-asyncssh_2.5.0-1_all.deb ... 2023-12-01T06:20:40.744 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-12-01T06:20:41.055 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-babel-localedata. 2023-12-01T06:20:41.070 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../063-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-12-01T06:20:41.113 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-12-01T06:20:42.043 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-babel. 2023-12-01T06:20:42.071 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../064-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-12-01T06:20:42.110 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-12-01T06:20:42.430 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jinja2. 2023-12-01T06:20:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../065-python3-jinja2_3.0.3-1_all.deb ... 2023-12-01T06:20:42.496 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-12-01T06:20:42.833 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-12-01T06:20:42.860 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../066-ceph-mgr-cephadm_18.0.0-7432-gbbeb098e-1jammy_all.deb ... 2023-12-01T06:20:42.899 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-cephadm (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:43.228 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-bcrypt. 2023-12-01T06:20:43.255 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../067-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-12-01T06:20:43.294 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-12-01T06:20:43.597 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-repoze.lru. 2023-12-01T06:20:43.624 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2023-12-01T06:20:43.680 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-12-01T06:20:43.992 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-routes. 2023-12-01T06:20:44.019 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../069-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-12-01T06:20:44.058 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-12-01T06:20:44.395 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-12-01T06:20:44.422 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../070-ceph-mgr-dashboard_18.0.0-7432-gbbeb098e-1jammy_all.deb ... 2023-12-01T06:20:44.461 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-dashboard (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:44.873 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-12-01T06:20:44.901 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../071-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-12-01T06:20:44.965 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-12-01T06:20:45.461 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-joblib. 2023-12-01T06:20:45.487 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../072-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-12-01T06:20:45.526 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-12-01T06:20:45.888 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-12-01T06:20:45.916 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../073-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-12-01T06:20:45.955 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-12-01T06:20:46.241 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-sklearn. 2023-12-01T06:20:46.268 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../074-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-12-01T06:20:46.307 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-12-01T06:20:46.983 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-12-01T06:20:47.010 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../075-ceph-mgr-diskprediction-local_18.0.0-7432-gbbeb098e-1jammy_all.deb ... 2023-12-01T06:20:47.066 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:48.028 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cachetools. 2023-12-01T06:20:48.045 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../076-python3-cachetools_5.0.0-1_all.deb ... 2023-12-01T06:20:48.086 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-12-01T06:20:48.388 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyasn1. 2023-12-01T06:20:48.416 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../077-python3-pyasn1_0.4.8-1_all.deb ... 2023-12-01T06:20:48.455 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-12-01T06:20:48.784 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-12-01T06:20:48.811 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../078-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-12-01T06:20:48.850 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-12-01T06:20:49.320 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rsa. 2023-12-01T06:20:49.347 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../079-python3-rsa_4.8-1_all.deb ... 2023-12-01T06:20:49.386 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rsa (4.8-1) ... 2023-12-01T06:20:49.692 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-google-auth. 2023-12-01T06:20:49.719 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../080-python3-google-auth_1.5.1-3_all.deb ... 2023-12-01T06:20:49.758 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-12-01T06:20:50.078 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-12-01T06:20:50.105 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-12-01T06:20:50.145 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-12-01T06:20:50.447 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-websocket. 2023-12-01T06:20:50.474 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../082-python3-websocket_1.2.3-1_all.deb ... 2023-12-01T06:20:50.514 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-12-01T06:20:50.944 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-kubernetes. 2023-12-01T06:20:50.971 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../083-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-12-01T06:20:51.002 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-12-01T06:20:51.650 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-12-01T06:20:51.666 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_18.0.0-7432-gbbeb098e-1jammy_all.deb ... 2023-12-01T06:20:51.700 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:51.986 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libonig5:amd64. 2023-12-01T06:20:52.013 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../085-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-12-01T06:20:52.044 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-12-01T06:20:52.372 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libjq1:amd64. 2023-12-01T06:20:52.400 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../086-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-12-01T06:20:52.439 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-12-01T06:20:52.700 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package jq. 2023-12-01T06:20:52.727 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../087-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-12-01T06:20:52.775 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-12-01T06:20:53.011 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package socat. 2023-12-01T06:20:53.027 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../088-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-12-01T06:20:53.069 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-12-01T06:20:53.505 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package xmlstarlet. 2023-12-01T06:20:53.522 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-12-01T06:20:53.555 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-12-01T06:20:53.910 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-test. 2023-12-01T06:20:53.937 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../090-ceph-test_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:53.976 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-test (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:56.256 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-volume. 2023-12-01T06:20:56.283 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../091-ceph-volume_18.0.0-7432-gbbeb098e-1jammy_all.deb ... 2023-12-01T06:20:56.322 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-volume (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:56.635 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libcephfs-dev. 2023-12-01T06:20:56.651 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../092-libcephfs-dev_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:20:56.693 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libcephfs-dev (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:20:57.013 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package nvme-cli. 2023-12-01T06:20:57.040 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../093-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-12-01T06:20:57.071 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-12-01T06:20:57.410 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-12-01T06:20:57.425 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../094-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-12-01T06:20:57.468 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-12-01T06:20:57.830 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-iniconfig. 2023-12-01T06:20:57.857 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../095-python3-iniconfig_1.1.1-2_all.deb ... 2023-12-01T06:20:57.896 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-12-01T06:20:58.199 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pastescript. 2023-12-01T06:20:58.226 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../096-python3-pastescript_2.0.2-4_all.deb ... 2023-12-01T06:20:58.265 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-12-01T06:20:58.586 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pluggy. 2023-12-01T06:20:58.599 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../097-python3-pluggy_0.13.0-7.1_all.deb ... 2023-12-01T06:20:58.644 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-12-01T06:20:58.930 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-psutil. 2023-12-01T06:20:58.946 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../098-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-12-01T06:20:58.980 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-12-01T06:20:59.333 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-py. 2023-12-01T06:20:59.360 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../099-python3-py_1.10.0-1_all.deb ... 2023-12-01T06:20:59.399 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-py (1.10.0-1) ... 2023-12-01T06:20:59.728 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pygments. 2023-12-01T06:20:59.756 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../100-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-12-01T06:20:59.795 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-12-01T06:21:00.240 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyinotify. 2023-12-01T06:21:00.256 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../101-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-12-01T06:21:00.290 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-12-01T06:21:00.585 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-toml. 2023-12-01T06:21:00.612 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../102-python3-toml_0.10.2-1_all.deb ... 2023-12-01T06:21:00.651 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-12-01T06:21:00.929 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pytest. 2023-12-01T06:21:00.956 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../103-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-12-01T06:21:00.995 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-12-01T06:21:01.324 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-simplejson. 2023-12-01T06:21:01.352 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../104-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-12-01T06:21:01.390 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-12-01T06:21:01.669 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-12-01T06:21:01.696 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../105-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-12-01T06:21:01.727 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-12-01T06:21:02.447 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package radosgw. 2023-12-01T06:21:02.476 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../106-radosgw_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:21:02.514 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking radosgw (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:03.242 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package rbd-fuse. 2023-12-01T06:21:03.258 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../107-rbd-fuse_18.0.0-7432-gbbeb098e-1jammy_amd64.deb ... 2023-12-01T06:21:03.309 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking rbd-fuse (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:03.604 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-12-01T06:21:03.863 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-12-01T06:21:03.989 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-12-01T06:21:04.254 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-12-01T06:21:04.594 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-12-01T06:21:05.090 INFO:teuthology.orchestra.run.smithi170.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-12-01T06:21:05.208 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up cephadm (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:05.820 INFO:teuthology.orchestra.run.smithi170.stdout:Adding system user cephadm....done 2023-12-01T06:21:05.958 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-12-01T06:21:06.509 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-12-01T06:21:06.753 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-12-01T06:21:06.879 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-12-01T06:21:07.130 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-12-01T06:21:07.389 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-12-01T06:21:07.540 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-py (1.10.0-1) ... 2023-12-01T06:21:07.859 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-12-01T06:21:08.219 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-12-01T06:21:08.470 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-12-01T06:21:08.754 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-12-01T06:21:09.006 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-12-01T06:21:09.265 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-12-01T06:21:09.527 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-ceph-argparse (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:09.892 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-12-01T06:21:09.993 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-12-01T06:21:10.236 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-12-01T06:21:10.387 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-12-01T06:21:10.747 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-12-01T06:21:10.873 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-12-01T06:21:11.132 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-12-01T06:21:11.392 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-12-01T06:21:11.701 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-12-01T06:21:12.378 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-12-01T06:21:12.637 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-12-01T06:21:12.764 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-12-01T06:21:12.890 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-12-01T06:21:13.174 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-12-01T06:21:13.559 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-12-01T06:21:13.810 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-12-01T06:21:14.186 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-12-01T06:21:14.488 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-12-01T06:21:14.847 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-12-01T06:21:15.090 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-12-01T06:21:15.208 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-ceph-common (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:15.526 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-idna (3.3-1) ... 2023-12-01T06:21:15.844 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-12-01T06:21:17.042 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-01T06:21:17.176 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-toml (0.10.2-1) ... 2023-12-01T06:21:17.444 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-12-01T06:21:17.570 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-12-01T06:21:17.696 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-12-01T06:21:17.997 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-12-01T06:21:18.240 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-12-01T06:21:18.474 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-01T06:21:18.609 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-12-01T06:21:18.910 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-12-01T06:21:19.170 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-12-01T06:21:19.429 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-12-01T06:21:19.697 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-12-01T06:21:19.931 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-12-01T06:21:20.191 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-12-01T06:21:20.325 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-12-01T06:21:20.793 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-12-01T06:21:20.919 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-12-01T06:21:21.204 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-12-01T06:21:21.505 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-12-01T06:21:21.789 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-12-01T06:21:22.165 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-12-01T06:21:22.401 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-12-01T06:21:22.527 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-12-01T06:21:22.920 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-12-01T06:21:23.179 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-portend (3.0.0-1) ... 2023-12-01T06:21:23.422 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-01T06:21:23.532 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-12-01T06:21:23.832 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-12-01T06:21:24.142 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-12-01T06:21:24.419 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-12-01T06:21:24.544 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rsa (4.8-1) ... 2023-12-01T06:21:24.820 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-12-01T06:21:25.096 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-12-01T06:21:25.348 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-12-01T06:21:25.724 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-12-01T06:21:26.025 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-12-01T06:21:26.361 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-12-01T06:21:26.487 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librados2 (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:26.613 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librgw2 (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:26.739 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libsqlite3-mod-ceph (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:26.865 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libcephfs2 (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:26.999 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libradosstriper1 (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:27.134 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-12-01T06:21:27.538 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librbd1 (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:27.656 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-modules-core (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:27.765 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-fuse (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:27.960 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-12-01T06:21:27.960 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-12-01T06:21:28.474 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libcephfs-dev (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:28.609 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rados (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:28.735 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-12-01T06:21:29.940 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rbd (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:30.091 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up rbd-fuse (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:30.225 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rgw (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:30.351 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cephfs (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:30.477 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-common (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:30.979 INFO:teuthology.orchestra.run.smithi170.stdout:Adding group ceph....done 2023-12-01T06:21:31.364 INFO:teuthology.orchestra.run.smithi170.stdout:Adding system user ceph....done 2023-12-01T06:21:31.532 INFO:teuthology.orchestra.run.smithi170.stdout:Setting system user ceph properties....done 2023-12-01T06:21:31.550 INFO:teuthology.orchestra.run.smithi170.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-12-01T06:21:31.686 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-12-01T06:21:32.025 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-12-01T06:21:32.504 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-test (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:32.622 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up radosgw (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:33.082 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-12-01T06:21:33.082 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-12-01T06:21:33.598 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-base (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:33.960 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-12-01T06:21:34.498 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mds (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:34.808 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-12-01T06:21:34.808 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-12-01T06:21:35.319 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:35.520 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-12-01T06:21:35.520 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-12-01T06:21:36.053 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-osd (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:36.356 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-12-01T06:21:36.356 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-12-01T06:21:36.838 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-k8sevents (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:36.981 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:37.132 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mon (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:37.341 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-12-01T06:21:37.342 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-12-01T06:21:37.842 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-cephadm (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:37.985 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:38.111 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-dashboard (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:38.328 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-volume (18.0.0-7432-gbbeb098e-1jammy) ... 2023-12-01T06:21:38.665 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-12-01T06:21:39.774 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-12-01T06:21:40.242 INFO:teuthology.orchestra.run.smithi170.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-12-01T06:21:40.245 DEBUG:teuthology.parallel:result is None 2023-12-01T06:21:40.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=bbeb098e76b9019fafbe1c7bd55e48604c324c89 2023-12-01T06:21:40.383 DEBUG:teuthology.orchestra.run.smithi170:> dpkg-query -W -f '${Version}' ceph 2023-12-01T06:21:40.401 INFO:teuthology.orchestra.run.smithi170.stdout:18.0.0-7432-gbbeb098e-1jammy 2023-12-01T06:21:40.401 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7432-gbbeb098e-1jammy 2023-12-01T06:21:40.402 INFO:teuthology.task.install:The correct ceph version 18.0.0-7432-gbbeb098e-1jammy is installed. 2023-12-01T06:21:40.402 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-01T06:21:40.402 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-12-01T06:21:40.402 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-01T06:21:40.456 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-01T06:21:40.457 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-12-01T06:21:40.457 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2023-12-01T06:21:40.512 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-01T06:21:40.568 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-01T06:21:40.568 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-12-01T06:21:40.568 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-01T06:21:40.624 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-01T06:21:40.683 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-01T06:21:40.683 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-12-01T06:21:40.683 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2023-12-01T06:21:40.735 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-01T06:21:40.787 INFO:teuthology.run_tasks:Running task workunit... 2023-12-01T06:21:40.802 INFO:tasks.workunit:Pulling workunits from ref bbeb098e76b9019fafbe1c7bd55e48604c324c89 2023-12-01T06:21:40.802 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-12-01T06:21:40.802 INFO:tasks.workunit:timeout=3h 2023-12-01T06:21:40.802 INFO:tasks.workunit:cleanup=True 2023-12-01T06:21:40.803 DEBUG:teuthology.orchestra.run.smithi170:> stat -- /home/ubuntu/cephtest/mnt.0 2023-12-01T06:21:40.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T06:21:40.833 INFO:teuthology.orchestra.run.smithi170.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-12-01T06:21:40.833 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -- /home/ubuntu/cephtest/mnt.0 2023-12-01T06:21:40.880 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2023-12-01T06:21:40.881 DEBUG:teuthology.orchestra.run.smithi170:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2023-12-01T06:21:40.929 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bbeb098e76b9019fafbe1c7bd55e48604c324c89 2023-12-01T06:21:40.978 INFO:tasks.workunit.client.0.smithi170.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-12-01T06:22:42.357 INFO:tasks.workunit.client.0.smithi170.stderr:Note: switching to 'bbeb098e76b9019fafbe1c7bd55e48604c324c89'. 2023-12-01T06:22:42.358 INFO:tasks.workunit.client.0.smithi170.stderr: 2023-12-01T06:22:42.358 INFO:tasks.workunit.client.0.smithi170.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-12-01T06:22:42.358 INFO:tasks.workunit.client.0.smithi170.stderr:changes and commit them, and you can discard any commits you make in this 2023-12-01T06:22:42.358 INFO:tasks.workunit.client.0.smithi170.stderr:state without impacting any branches by switching back to a branch. 2023-12-01T06:22:42.358 INFO:tasks.workunit.client.0.smithi170.stderr: 2023-12-01T06:22:42.358 INFO:tasks.workunit.client.0.smithi170.stderr:If you want to create a new branch to retain commits you create, you may 2023-12-01T06:22:42.358 INFO:tasks.workunit.client.0.smithi170.stderr:do so (now or later) by using -c with the switch command. Example: 2023-12-01T06:22:42.358 INFO:tasks.workunit.client.0.smithi170.stderr: 2023-12-01T06:22:42.358 INFO:tasks.workunit.client.0.smithi170.stderr: git switch -c 2023-12-01T06:22:42.358 INFO:tasks.workunit.client.0.smithi170.stderr: 2023-12-01T06:22:42.359 INFO:tasks.workunit.client.0.smithi170.stderr:Or undo this operation with: 2023-12-01T06:22:42.359 INFO:tasks.workunit.client.0.smithi170.stderr: 2023-12-01T06:22:42.359 INFO:tasks.workunit.client.0.smithi170.stderr: git switch - 2023-12-01T06:22:42.359 INFO:tasks.workunit.client.0.smithi170.stderr: 2023-12-01T06:22:42.359 INFO:tasks.workunit.client.0.smithi170.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-12-01T06:22:42.359 INFO:tasks.workunit.client.0.smithi170.stderr: 2023-12-01T06:22:42.359 INFO:tasks.workunit.client.0.smithi170.stderr:HEAD is now at bbeb098e76b osd/scrub: handling unexpected scrub requests 2023-12-01T06:22:42.360 DEBUG:teuthology.orchestra.run.smithi170:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-12-01T06:22:42.409 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-12-01T06:22:42.409 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-12-01T06:22:42.456 INFO:tasks.workunit:Running workunits matching scrub on client.0... 2023-12-01T06:22:42.456 INFO:tasks.workunit:Running workunit scrub/osd-mapper.sh... 2023-12-01T06:22:42.456 DEBUG:teuthology.orchestra.run.smithi170:workunit test scrub/osd-mapper.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bbeb098e76b9019fafbe1c7bd55e48604c324c89 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh 2023-12-01T06:22:42.571 INFO:tasks.workunit.client.0.smithi170.stderr:stty: 'standard input': Inappropriate ioctl for device 2023-12-01T06:22:42.575 INFO:tasks.workunit.client.0.smithi170.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2023-12-01T06:22:42.575 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2307: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2023-12-01T06:22:42.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2307: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2023-12-01T06:22:42.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2308: main: export PYTHONWARNINGS=ignore 2023-12-01T06:22:42.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2308: main: PYTHONWARNINGS=ignore 2023-12-01T06:22:42.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: main: export CEPH_CONF=/dev/null 2023-12-01T06:22:42.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: main: CEPH_CONF=/dev/null 2023-12-01T06:22:42.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2310: main: unset CEPH_ARGS 2023-12-01T06:22:42.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2312: main: local code 2023-12-01T06:22:42.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2313: main: run td/osd-mapper 2023-12-01T06:22:42.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:11: run: local dir=td/osd-mapper 2023-12-01T06:22:42.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:12: run: shift 2023-12-01T06:22:42.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:14: run: export CEPH_MON=127.0.0.1:7144 2023-12-01T06:22:42.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:14: run: CEPH_MON=127.0.0.1:7144 2023-12-01T06:22:42.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:15: run: export CEPH_ARGS 2023-12-01T06:22:42.576 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:16: run: uuidgen 2023-12-01T06:22:42.577 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:16: run: CEPH_ARGS+='--fsid=d95d2cc3-e2b2-446d-b931-8d97092cc017 --auth-supported=none ' 2023-12-01T06:22:42.577 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:17: run: CEPH_ARGS+='--mon-host=127.0.0.1:7144 ' 2023-12-01T06:22:42.577 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:19: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2023-12-01T06:22:42.577 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:20: run: set 2023-12-01T06:22:42.578 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:20: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2023-12-01T06:22:42.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:20: run: local funcs=TEST_truncated_sna_record 2023-12-01T06:22:42.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:21: run: for func in $funcs 2023-12-01T06:22:42.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:22: run: setup td/osd-mapper 2023-12-01T06:22:42.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-mapper 2023-12-01T06:22:42.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-mapper 2023-12-01T06:22:42.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-mapper 2023-12-01T06:22:42.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-12-01T06:22:42.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-mapper KILL 2023-12-01T06:22:42.581 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-12-01T06:22:42.581 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-12-01T06:22:42.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-12-01T06:22:42.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-12-01T06:22:42.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-12-01T06:22:42.582 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-12-01T06:22:42.582 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-12-01T06:22:42.583 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-12-01T06:22:42.583 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-12-01T06:22:42.585 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-12-01T06:22:42.585 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-12-01T06:22:42.585 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-12-01T06:22:42.585 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:22:42.585 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-12-01T06:22:42.586 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-12-01T06:22:42.586 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:22:42.586 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-12-01T06:22:42.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-12-01T06:22:42.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-mapper 2023-12-01T06:22:42.589 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-12-01T06:22:42.589 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:22:42.589 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.30922 2023-12-01T06:22:42.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.30922 2023-12-01T06:22:42.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-12-01T06:22:42.590 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-12-01T06:22:42.590 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-mapper 2023-12-01T06:22:42.591 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2023-12-01T06:22:42.591 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:22:42.591 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.30922 2023-12-01T06:22:42.591 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.30922 2023-12-01T06:22:42.593 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2023-12-01T06:22:42.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2023-12-01T06:22:42.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2023-12-01T06:22:42.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2023-12-01T06:22:42.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-mapper 1' TERM HUP INT 2023-12-01T06:22:42.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:23: run: TEST_truncated_sna_record td/osd-mapper 2023-12-01T06:22:42.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:45: TEST_truncated_sna_record: local dir=td/osd-mapper 2023-12-01T06:22:42.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:46: TEST_truncated_sna_record: cluster_conf=(['osds_num']='3' ['pgs_in_pool']='4' ['pool_name']='test') 2023-12-01T06:22:42.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:46: TEST_truncated_sna_record: local -A cluster_conf 2023-12-01T06:22:42.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:52: TEST_truncated_sna_record: local extr_dbg=3 2023-12-01T06:22:42.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:53: TEST_truncated_sna_record: (( extr_dbg > 1 )) 2023-12-01T06:22:42.594 INFO:tasks.workunit.client.0.smithi170.stdout:Dir: td/osd-mapper 2023-12-01T06:22:42.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:53: TEST_truncated_sna_record: echo 'Dir: td/osd-mapper' 2023-12-01T06:22:42.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:54: TEST_truncated_sna_record: standard_scrub_cluster td/osd-mapper cluster_conf 2023-12-01T06:22:42.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:221: standard_scrub_cluster: local dir=td/osd-mapper 2023-12-01T06:22:42.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:222: standard_scrub_cluster: local -n args=cluster_conf 2023-12-01T06:22:42.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:224: standard_scrub_cluster: local OSDS=3 2023-12-01T06:22:42.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:225: standard_scrub_cluster: local pg_num=4 2023-12-01T06:22:42.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:226: standard_scrub_cluster: local poolname=test 2023-12-01T06:22:42.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:227: standard_scrub_cluster: args['pool_name']=test 2023-12-01T06:22:42.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:228: standard_scrub_cluster: local extra_pars= 2023-12-01T06:22:42.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:229: standard_scrub_cluster: local debug_msg=dbg 2023-12-01T06:22:42.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:232: standard_scrub_cluster: local saved_echo_flag=x 2023-12-01T06:22:42.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:233: standard_scrub_cluster: set +x 2023-12-01T06:22:44.142 INFO:tasks.workunit.client.0.smithi170.stdout:add osd0 805ef592-ce40-4fba-84bd-bd7db4bfd14d 2023-12-01T06:22:44.415 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:22:44.518 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:22:44.515+0000 7f900750a640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:22:44.561 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:22:44.559+0000 7f900750a640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:22:44.562 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:22:44.559+0000 7f900750a640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:22:44.563 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:22:44.563+0000 7f900750a640 -1 bluestore(td/osd-mapper/0/block) _read_bdev_label failed to open td/osd-mapper/0/block: (2) No such file or directory 2023-12-01T06:22:44.563 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:22:44.563+0000 7f900750a640 -1 bluestore(td/osd-mapper/0/block) _read_bdev_label failed to open td/osd-mapper/0/block: (2) No such file or directory 2023-12-01T06:22:44.563 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:22:44.563+0000 7f900750a640 -1 bluestore(td/osd-mapper/0/block) _read_bdev_label failed to open td/osd-mapper/0/block: (2) No such file or directory 2023-12-01T06:22:44.600 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:22:44.599+0000 7f900750a640 -1 bluestore(td/osd-mapper/0) _read_fsid unparsable uuid 2023-12-01T06:22:50.663 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd0 key to auth repository 2023-12-01T06:22:50.855 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.0 2023-12-01T06:22:50.883 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:22:50.883+0000 7f7b7169a640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:22:50.886 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:22:50.883+0000 7f7b7169a640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:22:50.887 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:22:50.887+0000 7f7b7169a640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:22:51.065 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:22:51.436 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:22:51.435+0000 7f7b7169a640 -1 Falling back to public interface 2023-12-01T06:22:52.265 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:22:53.452 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:22:54.072 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:22:54.071+0000 7f7b7169a640 -1 osd.0 0 log_to_monitors true 2023-12-01T06:22:54.754 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:22:56.247 INFO:tasks.workunit.client.0.smithi170.stdout:4 2023-12-01T06:22:57.560 INFO:tasks.workunit.client.0.smithi170.stdout:5 2023-12-01T06:22:58.948 INFO:tasks.workunit.client.0.smithi170.stdout:6 2023-12-01T06:23:00.254 INFO:tasks.workunit.client.0.smithi170.stdout:7 2023-12-01T06:23:01.592 INFO:tasks.workunit.client.0.smithi170.stdout:8 2023-12-01T06:23:02.904 INFO:tasks.workunit.client.0.smithi170.stdout:9 2023-12-01T06:23:04.216 INFO:tasks.workunit.client.0.smithi170.stdout:10 2023-12-01T06:23:05.587 INFO:tasks.workunit.client.0.smithi170.stdout:11 2023-12-01T06:23:07.171 INFO:tasks.workunit.client.0.smithi170.stdout:12 2023-12-01T06:23:08.741 INFO:tasks.workunit.client.0.smithi170.stdout:13 2023-12-01T06:23:10.393 INFO:tasks.workunit.client.0.smithi170.stdout:14 2023-12-01T06:23:11.798 INFO:tasks.workunit.client.0.smithi170.stdout:15 2023-12-01T06:23:13.951 INFO:tasks.workunit.client.0.smithi170.stdout:16 2023-12-01T06:23:15.262 INFO:tasks.workunit.client.0.smithi170.stdout:17 2023-12-01T06:23:16.738 INFO:tasks.workunit.client.0.smithi170.stdout:18 2023-12-01T06:23:18.183 INFO:tasks.workunit.client.0.smithi170.stdout:19 2023-12-01T06:23:18.510 INFO:tasks.workunit.client.0.smithi170.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3099783336,v1:127.0.0.1:6801/3099783336] [v2:127.0.0.1:6802/3099783336,v1:127.0.0.1:6803/3099783336] exists,up 805ef592-ce40-4fba-84bd-bd7db4bfd14d 2023-12-01T06:23:18.515 INFO:tasks.workunit.client.0.smithi170.stdout:add osd1 28542883-7e5e-4a91-9830-a734fb6ad0ff 2023-12-01T06:23:18.869 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:23:18.914 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:18.910+0000 7f764a1c1640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:23:18.945 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:18.942+0000 7f764a1c1640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:23:18.947 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:18.942+0000 7f764a1c1640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:23:18.947 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:18.946+0000 7f764a1c1640 -1 bluestore(td/osd-mapper/1/block) _read_bdev_label failed to open td/osd-mapper/1/block: (2) No such file or directory 2023-12-01T06:23:18.948 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:18.946+0000 7f764a1c1640 -1 bluestore(td/osd-mapper/1/block) _read_bdev_label failed to open td/osd-mapper/1/block: (2) No such file or directory 2023-12-01T06:23:18.948 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:18.946+0000 7f764a1c1640 -1 bluestore(td/osd-mapper/1/block) _read_bdev_label failed to open td/osd-mapper/1/block: (2) No such file or directory 2023-12-01T06:23:19.007 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:19.002+0000 7f764a1c1640 -1 bluestore(td/osd-mapper/1) _read_fsid unparsable uuid 2023-12-01T06:23:25.136 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd1 key to auth repository 2023-12-01T06:23:25.549 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.1 2023-12-01T06:23:25.583 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:25.577+0000 7f6d5f178640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:23:25.619 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:25.613+0000 7f6d5f178640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:23:25.620 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:25.617+0000 7f6d5f178640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:23:25.883 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:23:26.181 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:26.177+0000 7f6d5f178640 -1 Falling back to public interface 2023-12-01T06:23:27.196 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:23:28.494 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:23:28.715 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:28.709+0000 7f6d5f178640 -1 osd.1 0 log_to_monitors true 2023-12-01T06:23:29.809 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:23:31.121 INFO:tasks.workunit.client.0.smithi170.stdout:4 2023-12-01T06:23:32.487 INFO:tasks.workunit.client.0.smithi170.stdout:5 2023-12-01T06:23:33.800 INFO:tasks.workunit.client.0.smithi170.stdout:6 2023-12-01T06:23:35.106 INFO:tasks.workunit.client.0.smithi170.stdout:7 2023-12-01T06:23:36.508 INFO:tasks.workunit.client.0.smithi170.stdout:8 2023-12-01T06:23:37.818 INFO:tasks.workunit.client.0.smithi170.stdout:9 2023-12-01T06:23:39.133 INFO:tasks.workunit.client.0.smithi170.stdout:10 2023-12-01T06:23:40.508 INFO:tasks.workunit.client.0.smithi170.stdout:11 2023-12-01T06:23:42.164 INFO:tasks.workunit.client.0.smithi170.stdout:12 2023-12-01T06:23:43.531 INFO:tasks.workunit.client.0.smithi170.stdout:13 2023-12-01T06:23:45.003 INFO:tasks.workunit.client.0.smithi170.stdout:14 2023-12-01T06:23:46.861 INFO:tasks.workunit.client.0.smithi170.stdout:15 2023-12-01T06:23:48.606 INFO:tasks.workunit.client.0.smithi170.stdout:16 2023-12-01T06:23:50.481 INFO:tasks.workunit.client.0.smithi170.stdout:17 2023-12-01T06:23:51.818 INFO:tasks.workunit.client.0.smithi170.stdout:18 2023-12-01T06:23:52.179 INFO:tasks.workunit.client.0.smithi170.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1749353292,v1:127.0.0.1:6811/1749353292] [v2:127.0.0.1:6812/1749353292,v1:127.0.0.1:6813/1749353292] exists,up 28542883-7e5e-4a91-9830-a734fb6ad0ff 2023-12-01T06:23:52.186 INFO:tasks.workunit.client.0.smithi170.stdout:add osd2 9a2447a8-77da-4cdc-85fd-367ebd30bf9b 2023-12-01T06:23:52.529 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:23:52.582 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:52.579+0000 7fbd8396d640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:23:52.596 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:52.591+0000 7fbd8396d640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:23:52.597 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:52.595+0000 7fbd8396d640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:23:52.598 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:52.595+0000 7fbd8396d640 -1 bluestore(td/osd-mapper/2/block) _read_bdev_label failed to open td/osd-mapper/2/block: (2) No such file or directory 2023-12-01T06:23:52.598 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:52.595+0000 7fbd8396d640 -1 bluestore(td/osd-mapper/2/block) _read_bdev_label failed to open td/osd-mapper/2/block: (2) No such file or directory 2023-12-01T06:23:52.598 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:52.595+0000 7fbd8396d640 -1 bluestore(td/osd-mapper/2/block) _read_bdev_label failed to open td/osd-mapper/2/block: (2) No such file or directory 2023-12-01T06:23:52.659 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:52.655+0000 7fbd8396d640 -1 bluestore(td/osd-mapper/2) _read_fsid unparsable uuid 2023-12-01T06:23:59.009 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd2 key to auth repository 2023-12-01T06:23:59.384 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.2 2023-12-01T06:23:59.417 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:59.415+0000 7fe0bd9f6640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:23:59.438 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:59.435+0000 7fe0bd9f6640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:23:59.439 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:23:59.435+0000 7fe0bd9f6640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:23:59.675 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:24:00.015 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:00.011+0000 7fe0bd9f6640 -1 Falling back to public interface 2023-12-01T06:24:00.972 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:24:02.311 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:24:02.572 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:02.567+0000 7fe0bd9f6640 -1 osd.2 0 log_to_monitors true 2023-12-01T06:24:03.622 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:24:04.933 INFO:tasks.workunit.client.0.smithi170.stdout:4 2023-12-01T06:24:06.364 INFO:tasks.workunit.client.0.smithi170.stdout:5 2023-12-01T06:24:07.674 INFO:tasks.workunit.client.0.smithi170.stdout:6 2023-12-01T06:24:08.976 INFO:tasks.workunit.client.0.smithi170.stdout:7 2023-12-01T06:24:10.396 INFO:tasks.workunit.client.0.smithi170.stdout:8 2023-12-01T06:24:11.708 INFO:tasks.workunit.client.0.smithi170.stdout:9 2023-12-01T06:24:13.019 INFO:tasks.workunit.client.0.smithi170.stdout:10 2023-12-01T06:24:14.377 INFO:tasks.workunit.client.0.smithi170.stdout:11 2023-12-01T06:24:16.049 INFO:tasks.workunit.client.0.smithi170.stdout:12 2023-12-01T06:24:17.646 INFO:tasks.workunit.client.0.smithi170.stdout:13 2023-12-01T06:24:19.189 INFO:tasks.workunit.client.0.smithi170.stdout:14 2023-12-01T06:24:20.590 INFO:tasks.workunit.client.0.smithi170.stdout:15 2023-12-01T06:24:22.401 INFO:tasks.workunit.client.0.smithi170.stdout:16 2023-12-01T06:24:23.801 INFO:tasks.workunit.client.0.smithi170.stdout:17 2023-12-01T06:24:25.583 INFO:tasks.workunit.client.0.smithi170.stdout:18 2023-12-01T06:24:25.934 INFO:tasks.workunit.client.0.smithi170.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3477449825,v1:127.0.0.1:6819/3477449825] [v2:127.0.0.1:6820/3477449825,v1:127.0.0.1:6821/3477449825] exists,up 9a2447a8-77da-4cdc-85fd-367ebd30bf9b 2023-12-01T06:24:26.327 INFO:tasks.workunit.client.0.smithi170.stderr:pool 'test' created 2023-12-01T06:24:28.469 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.0 seq 21474836500 2023-12-01T06:24:28.774 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.1 seq 42949672975 2023-12-01T06:24:29.092 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.2 seq 64424509447 2023-12-01T06:24:33.084 INFO:tasks.workunit.client.0.smithi170.stdout:standard_scrub_cluster: dbg: test pool is test 1 2023-12-01T06:24:33.084 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:55: TEST_truncated_sna_record: ceph tell 'osd.*' config set osd_stats_update_period_not_scrubbing 1 2023-12-01T06:24:33.236 INFO:tasks.workunit.client.0.smithi170.stdout:osd.0: { 2023-12-01T06:24:33.236 INFO:tasks.workunit.client.0.smithi170.stdout: "success": "osd_stats_update_period_not_scrubbing = '' (not observed, change may require restart) " 2023-12-01T06:24:33.236 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:24:33.249 INFO:tasks.workunit.client.0.smithi170.stdout:osd.1: { 2023-12-01T06:24:33.249 INFO:tasks.workunit.client.0.smithi170.stdout: "success": "osd_stats_update_period_not_scrubbing = '' (not observed, change may require restart) " 2023-12-01T06:24:33.249 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:24:33.262 INFO:tasks.workunit.client.0.smithi170.stdout:osd.2: { 2023-12-01T06:24:33.262 INFO:tasks.workunit.client.0.smithi170.stdout: "success": "osd_stats_update_period_not_scrubbing = '' (not observed, change may require restart) " 2023-12-01T06:24:33.262 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:24:33.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:56: TEST_truncated_sna_record: ceph tell 'osd.*' config set osd_stats_update_period_scrubbing 1 2023-12-01T06:24:33.402 INFO:tasks.workunit.client.0.smithi170.stdout:osd.0: { 2023-12-01T06:24:33.403 INFO:tasks.workunit.client.0.smithi170.stdout: "success": "osd_stats_update_period_scrubbing = '' (not observed, change may require restart) " 2023-12-01T06:24:33.403 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:24:33.416 INFO:tasks.workunit.client.0.smithi170.stdout:osd.1: { 2023-12-01T06:24:33.416 INFO:tasks.workunit.client.0.smithi170.stdout: "success": "osd_stats_update_period_scrubbing = '' (not observed, change may require restart) " 2023-12-01T06:24:33.416 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:24:33.429 INFO:tasks.workunit.client.0.smithi170.stdout:osd.2: { 2023-12-01T06:24:33.429 INFO:tasks.workunit.client.0.smithi170.stdout: "success": "osd_stats_update_period_scrubbing = '' (not observed, change may require restart) " 2023-12-01T06:24:33.429 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:24:33.444 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:58: TEST_truncated_sna_record: local osdn=3 2023-12-01T06:24:33.444 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:59: TEST_truncated_sna_record: local poolid=1 2023-12-01T06:24:33.444 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:60: TEST_truncated_sna_record: local poolname=test 2023-12-01T06:24:33.444 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:61: TEST_truncated_sna_record: local objname=objxxx 2023-12-01T06:24:33.444 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:64: TEST_truncated_sna_record: make_a_clone test objxxx snap01 snap02 2023-12-01T06:24:33.444 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:32: make_a_clone: local saved_echo_flag=x 2023-12-01T06:24:33.444 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:33: make_a_clone: set +x 2023-12-01T06:24:33.721 INFO:tasks.workunit.client.0.smithi170.stdout:created pool test snap snap01 2023-12-01T06:24:33.934 INFO:tasks.workunit.client.0.smithi170.stdout:created pool test snap snap02 2023-12-01T06:24:33.938 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:65: TEST_truncated_sna_record: make_a_clone test objxxx snap13 2023-12-01T06:24:33.938 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:32: make_a_clone: local saved_echo_flag=x 2023-12-01T06:24:33.939 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:33: make_a_clone: set +x 2023-12-01T06:24:34.409 INFO:tasks.workunit.client.0.smithi170.stdout:created pool test snap snap13 2023-12-01T06:24:34.413 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:66: TEST_truncated_sna_record: make_a_clone test objxxx snap24 snap25 2023-12-01T06:24:34.413 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:32: make_a_clone: local saved_echo_flag=x 2023-12-01T06:24:34.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:33: make_a_clone: set +x 2023-12-01T06:24:34.811 INFO:tasks.workunit.client.0.smithi170.stdout:created pool test snap snap24 2023-12-01T06:24:35.008 INFO:tasks.workunit.client.0.smithi170.stdout:created pool test snap snap25 2023-12-01T06:24:35.012 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:67: TEST_truncated_sna_record: echo 18562 2023-12-01T06:24:35.012 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:67: TEST_truncated_sna_record: rados -p test put objxxx - 2023-12-01T06:24:35.323 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:70: TEST_truncated_sna_record: ceph --format=json-pretty osd map test objxxx 2023-12-01T06:24:35.323 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:70: TEST_truncated_sna_record: jq -r .pgid 2023-12-01T06:24:35.641 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:70: TEST_truncated_sna_record: local pgid=1.3 2023-12-01T06:24:35.642 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:71: TEST_truncated_sna_record: ceph --format=json-pretty osd map test objxxx 2023-12-01T06:24:35.642 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:71: TEST_truncated_sna_record: jq -r '.up[0]' 2023-12-01T06:24:35.967 INFO:tasks.workunit.client.0.smithi170.stdout:pgid is 1.3 (primary: osd.1) 2023-12-01T06:24:35.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:71: TEST_truncated_sna_record: local osd=1 2023-12-01T06:24:35.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:72: TEST_truncated_sna_record: echo 'pgid is 1.3 (primary: osd.1)' 2023-12-01T06:24:35.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:74: TEST_truncated_sna_record: set_query_debug 1.3 2023-12-01T06:24:35.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:294: set_query_debug: local pgid=1.3 2023-12-01T06:24:35.968 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:296: set_query_debug: awk -v 'pg=^1.3' -n -e '$0 ~ pg { print(gensub(/[^0-9]*([0-9]+).*/,"\\1","g",$5)); }' 2023-12-01T06:24:35.969 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:296: set_query_debug: ceph pg dump pgs_brief 2023-12-01T06:24:36.272 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs_brief 2023-12-01T06:24:36.289 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:296: set_query_debug: local prim_osd=1 2023-12-01T06:24:36.289 INFO:tasks.workunit.client.0.smithi170.stdout:Setting scrub debug data. Primary for 1.3 is 1 2023-12-01T06:24:36.290 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:298: set_query_debug: echo 'Setting scrub debug data. Primary for 1.3 is 1' 2023-12-01T06:24:36.291 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:299: set_query_debug: get_asok_path osd.1 2023-12-01T06:24:36.291 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:24:36.291 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:24:36.291 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:24:36.291 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:24:36.291 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.30922 2023-12-01T06:24:36.291 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.30922/ceph-osd.1.asok 2023-12-01T06:24:36.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:299: set_query_debug: CEPH_ARGS= 2023-12-01T06:24:36.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:299: set_query_debug: ceph --format=json daemon /tmp/ceph-asok.30922/ceph-osd.1.asok scrubdebug 1.3 set sessions 2023-12-01T06:24:36.418 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:77: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2023-12-01T06:24:36.418 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:77: TEST_truncated_sna_record: rados --format json-pretty -p test listsnaps objxxx 2023-12-01T06:24:36.478 INFO:tasks.workunit.client.0.smithi170.stdout:{"success":true}{ 2023-12-01T06:24:36.478 INFO:tasks.workunit.client.0.smithi170.stdout: "name": "objxxx", 2023-12-01T06:24:36.478 INFO:tasks.workunit.client.0.smithi170.stdout: "clones": [ 2023-12-01T06:24:36.478 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:24:36.478 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 2, 2023-12-01T06:24:36.478 INFO:tasks.workunit.client.0.smithi170.stdout: "snapshots": [ 2023-12-01T06:24:36.478 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 1, 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: "name": "snap01" 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: }, 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 2, 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: "name": "snap02" 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: } 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: ], 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: "size": 6, 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: "overlaps": [] 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: }, 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 3, 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: "snapshots": [ 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:24:36.479 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 3, 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: "name": "snap13" 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: } 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: ], 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: "size": 6, 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: "overlaps": [] 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: }, 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 5, 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: "snapshots": [ 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 4, 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: "name": "snap24" 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: }, 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:24:36.480 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 5, 2023-12-01T06:24:36.481 INFO:tasks.workunit.client.0.smithi170.stdout: "name": "snap25" 2023-12-01T06:24:36.481 INFO:tasks.workunit.client.0.smithi170.stdout: } 2023-12-01T06:24:36.481 INFO:tasks.workunit.client.0.smithi170.stdout: ], 2023-12-01T06:24:36.481 INFO:tasks.workunit.client.0.smithi170.stdout: "size": 6, 2023-12-01T06:24:36.481 INFO:tasks.workunit.client.0.smithi170.stdout: "overlaps": [] 2023-12-01T06:24:36.481 INFO:tasks.workunit.client.0.smithi170.stdout: }, 2023-12-01T06:24:36.481 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:24:36.481 INFO:tasks.workunit.client.0.smithi170.stdout: "id": "head", 2023-12-01T06:24:36.481 INFO:tasks.workunit.client.0.smithi170.stdout: "snapshots": [], 2023-12-01T06:24:36.481 INFO:tasks.workunit.client.0.smithi170.stdout: "size": 6 2023-12-01T06:24:36.481 INFO:tasks.workunit.client.0.smithi170.stdout: } 2023-12-01T06:24:36.481 INFO:tasks.workunit.client.0.smithi170.stdout: ] 2023-12-01T06:24:36.481 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:24:36.482 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:80: TEST_truncated_sna_record: ceph pg 1.3 deep-scrub 2023-12-01T06:24:36.608 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:24:36.608 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": true, 2023-12-01T06:24:36.608 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:24:36.608 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:24:36.608 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:24:36.622 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:84: TEST_truncated_sna_record: sleep 3 2023-12-01T06:24:39.623 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:85: TEST_truncated_sna_record: ceph pg dump pgs 2023-12-01T06:24:39.912 INFO:tasks.workunit.client.0.smithi170.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2023-12-01T06:24:39.912 INFO:tasks.workunit.client.0.smithi170.stdout:1.3 4 0 0 0 0 24 0 0 7 0 7 active+clean 2023-12-01T06:24:27.571678+0000 26'7 26:27 [1,2,0] 1 [1,2,0] 1 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:24:39.912 INFO:tasks.workunit.client.0.smithi170.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:24:27.518384+0000 0'0 21:15 [0,1,2] 0 [0,1,2] 0 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:24:39.912 INFO:tasks.workunit.client.0.smithi170.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:24:27.341553+0000 0'0 26:24 [2,0,1] 2 [2,0,1] 2 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:24:39.912 INFO:tasks.workunit.client.0.smithi170.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:24:27.571348+0000 0'0 26:23 [1,0,2] 1 [1,0,2] 1 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:24:39.912 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:39.912 INFO:tasks.workunit.client.0.smithi170.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2023-12-01T06:24:39.912 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:24:39.928 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:86: TEST_truncated_sna_record: grep -a -q -- 'event: --^^^^---- ScrubFinished' td/osd-mapper/osd.1.log 2023-12-01T06:24:39.931 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:90: TEST_truncated_sna_record: ceph pg dump pgs 2023-12-01T06:24:40.219 INFO:tasks.workunit.client.0.smithi170.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2023-12-01T06:24:40.219 INFO:tasks.workunit.client.0.smithi170.stdout:1.3 4 0 0 0 0 24 0 0 7 0 7 active+clean 2023-12-01T06:24:27.571678+0000 26'7 26:27 [1,2,0] 1 [1,2,0] 1 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:24:40.219 INFO:tasks.workunit.client.0.smithi170.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:24:27.518384+0000 0'0 21:15 [0,1,2] 0 [0,1,2] 0 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:24:40.219 INFO:tasks.workunit.client.0.smithi170.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:24:27.341553+0000 0'0 26:24 [2,0,1] 2 [2,0,1] 2 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:24:40.219 INFO:tasks.workunit.client.0.smithi170.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:24:27.571348+0000 0'0 26:23 [1,0,2] 1 [1,0,2] 1 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:24:40.219 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:40.219 INFO:tasks.workunit.client.0.smithi170.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2023-12-01T06:24:40.220 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:24:40.235 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:91: TEST_truncated_sna_record: ceph osd set noscrub 2023-12-01T06:24:40.667 INFO:tasks.workunit.client.0.smithi170.stderr:noscrub is set 2023-12-01T06:24:40.697 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:92: TEST_truncated_sna_record: ceph osd set nodeep-scrub 2023-12-01T06:24:41.253 INFO:tasks.workunit.client.0.smithi170.stderr:nodeep-scrub is set 2023-12-01T06:24:41.277 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:93: TEST_truncated_sna_record: sleep 5 2023-12-01T06:24:46.279 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:94: TEST_truncated_sna_record: grep -a -q -v ERR td/osd-mapper/osd.1.log 2023-12-01T06:24:46.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:97: TEST_truncated_sna_record: kill_daemons td/osd-mapper TERM osd 2023-12-01T06:24:46.283 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-12-01T06:24:46.283 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-12-01T06:24:46.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-12-01T06:24:46.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-12-01T06:24:46.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-12-01T06:24:46.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-12-01T06:24:46.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:99: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2023-12-01T06:24:46.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:99: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 dump p 2023-12-01T06:24:46.625 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.618+0000 7f0d457df940 1 bdev(0x55bd4292e000 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2023-12-01T06:24:46.625 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.618+0000 7f0d457df940 1 bdev(0x55bd4292e000 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:24:46.625 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.618+0000 7f0d457df940 1 bluestore(td/osd-mapper/0) _set_cache_sizes cache_size 1073741824 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2023-12-01T06:24:46.625 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.618+0000 7f0d457df940 1 bdev(0x55bd4292e700 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2023-12-01T06:24:46.625 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.618+0000 7f0d457df940 1 bdev(0x55bd4292e700 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:24:46.625 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.618+0000 7f0d457df940 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2023-12-01T06:24:46.625 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.618+0000 7f0d457df940 1 bluefs mount 2023-12-01T06:24:46.632 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.626+0000 7f0d457df940 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2023-12-01T06:24:46.659 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.654+0000 7f0d457df940 1 bluefs mount shared_bdev_used = 0 2023-12-01T06:24:46.659 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.654+0000 7f0d457df940 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2023-12-01T06:24:46.666 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.662+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2023-12-01T06:24:46.667 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.662+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2023-12-01T06:24:46.667 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.662+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2023-12-01T06:24:46.667 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.662+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2023-12-01T06:24:46.667 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.662+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2023-12-01T06:24:46.668 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.662+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2023-12-01T06:24:46.668 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.662+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2023-12-01T06:24:46.668 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.662+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2023-12-01T06:24:46.669 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.662+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2023-12-01T06:24:46.669 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.662+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2023-12-01T06:24:46.669 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.662+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2023-12-01T06:24:46.669 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.662+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2023-12-01T06:24:46.859 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.854+0000 7f0d457df940 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2023-12-01T06:24:46.859 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.854+0000 7f0d457df940 1 bluestore(td/osd-mapper/0) _open_super_meta old nid_max 2050 2023-12-01T06:24:46.860 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.854+0000 7f0d457df940 1 bluestore(td/osd-mapper/0) _open_super_meta old blobid_max 20480 2023-12-01T06:24:46.860 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.854+0000 7f0d457df940 1 bluestore(td/osd-mapper/0) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2023-12-01T06:24:46.860 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.854+0000 7f0d457df940 1 bluestore(td/osd-mapper/0) _open_super_meta min_alloc_size 0x1000 2023-12-01T06:24:46.860 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.854+0000 7f0d457df940 1 freelist init 2023-12-01T06:24:46.860 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.854+0000 7f0d457df940 1 freelist _read_cfg 2023-12-01T06:24:46.864 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.858+0000 7f0d457df940 1 bluestore(td/osd-mapper/0) _init_alloc loaded 100 GiB in 24 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc0000, fragmentation 8.77383e-07 2023-12-01T06:24:46.865 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.858+0000 7f0d457df940 1 bluefs umount 2023-12-01T06:24:46.865 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.858+0000 7f0d457df940 1 bdev(0x55bd4292e700 td/osd-mapper/0/block) close 2023-12-01T06:24:46.905 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.902+0000 7f0d457df940 1 bdev(0x55bd4292e700 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2023-12-01T06:24:46.905 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.902+0000 7f0d457df940 1 bdev(0x55bd4292e700 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:24:46.905 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.902+0000 7f0d457df940 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2023-12-01T06:24:46.905 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.902+0000 7f0d457df940 1 bluefs mount 2023-12-01T06:24:46.906 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.902+0000 7f0d457df940 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2023-12-01T06:24:46.912 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.906+0000 7f0d457df940 1 bluefs mount shared_bdev_used = 27459584 2023-12-01T06:24:46.912 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.906+0000 7f0d457df940 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2023-12-01T06:24:46.920 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.914+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2023-12-01T06:24:46.920 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.914+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2023-12-01T06:24:46.920 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.914+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2023-12-01T06:24:46.920 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.914+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2023-12-01T06:24:46.920 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.914+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2023-12-01T06:24:46.920 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.914+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2023-12-01T06:24:46.921 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.914+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2023-12-01T06:24:46.921 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.914+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2023-12-01T06:24:46.921 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.914+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2023-12-01T06:24:46.921 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.914+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2023-12-01T06:24:46.921 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.914+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2023-12-01T06:24:46.922 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:46.918+0000 7f0d457df940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2023-12-01T06:24:48.063 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.058+0000 7f0d457df940 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2023-12-01T06:24:48.068 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2023-12-01T06:24:48.069 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 0b 05 58 02 00 00 d9 5d 2c c3 e2 b2 44 6d b9 31 |..X....],...Dm.1| 2023-12-01T06:24:48.069 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 8d 97 09 2c c0 17 00 00 00 00 1c 00 00 00 00 00 |...,............| 2023-12-01T06:24:48.069 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-12-01T06:24:48.069 INFO:tasks.workunit.client.0.smithi170.stdout:* 2023-12-01T06:24:48.069 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2023-12-01T06:24:48.069 INFO:tasks.workunit.client.0.smithi170.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2023-12-01T06:24:48.069 INFO:tasks.workunit.client.0.smithi170.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2023-12-01T06:24:48.069 INFO:tasks.workunit.client.0.smithi170.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2023-12-01T06:24:48.069 INFO:tasks.workunit.client.0.smithi170.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2023-12-01T06:24:48.069 INFO:tasks.workunit.client.0.smithi170.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2023-12-01T06:24:48.069 INFO:tasks.workunit.client.0.smithi170.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2023-12-01T06:24:48.069 INFO:tasks.workunit.client.0.smithi170.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2023-12-01T06:24:48.069 INFO:tasks.workunit.client.0.smithi170.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2023-12-01T06:24:48.069 INFO:tasks.workunit.client.0.smithi170.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2023-12-01T06:24:48.070 INFO:tasks.workunit.client.0.smithi170.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2023-12-01T06:24:48.071 INFO:tasks.workunit.client.0.smithi170.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2023-12-01T06:24:48.071 INFO:tasks.workunit.client.0.smithi170.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2023-12-01T06:24:48.071 INFO:tasks.workunit.client.0.smithi170.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2023-12-01T06:24:48.071 INFO:tasks.workunit.client.0.smithi170.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2023-12-01T06:24:48.071 INFO:tasks.workunit.client.0.smithi170.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2023-12-01T06:24:48.071 INFO:tasks.workunit.client.0.smithi170.stdout:00000220 72 65 1c 00 00 00 05 00 00 00 80 5e f5 92 ce 40 |re.........^...@| 2023-12-01T06:24:48.071 INFO:tasks.workunit.client.0.smithi170.stdout:00000230 4f ba 84 bd bd 7d b4 bf d1 4d 00 00 00 00 00 00 |O....}...M......| 2023-12-01T06:24:48.071 INFO:tasks.workunit.client.0.smithi170.stdout:00000240 00 00 1c 00 00 00 d5 7b 69 65 13 e6 cb 07 01 00 |.......{ie......| 2023-12-01T06:24:48.071 INFO:tasks.workunit.client.0.smithi170.stdout:00000250 00 00 01 00 00 00 01 00 00 00 1c 00 00 00 |..............| 2023-12-01T06:24:48.071 INFO:tasks.workunit.client.0.smithi170.stdout:0000025e 2023-12-01T06:24:48.071 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:48.071 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2023-12-01T06:24:48.072 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:48.072 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:24:48.072 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2023-12-01T06:24:48.072 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2023-12-01T06:24:48.072 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:24:48.072 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2023-12-01T06:24:48.072 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 00 |.......| 2023-12-01T06:24:48.072 INFO:tasks.workunit.client.0.smithi170.stdout:00000047 2023-12-01T06:24:48.072 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:48.072 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:24:48.072 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2023-12-01T06:24:48.072 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2023-12-01T06:24:48.072 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:24:48.072 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout:0000003f 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 00 |.......| 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout:00000047 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:48.073 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:48.074 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:48.075 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2023-12-01T06:24:48.076 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:48.076 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2023-12-01T06:24:48.076 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:48.076 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.062+0000 7f0d457df940 1 bluefs umount 2023-12-01T06:24:48.076 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.062+0000 7f0d457df940 1 bdev(0x55bd4292e700 td/osd-mapper/0/block) close 2023-12-01T06:24:48.317 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.314+0000 7f0d457df940 1 freelist shutdown 2023-12-01T06:24:48.317 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.314+0000 7f0d457df940 1 bdev(0x55bd4292e000 td/osd-mapper/0/block) close 2023-12-01T06:24:48.413 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:100: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2023-12-01T06:24:48.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:100: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2023-12-01T06:24:48.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:100: TEST_truncated_sna_record: grep -a SNA_ 2023-12-01T06:24:48.444 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.438+0000 7ffae66bb940 1 bdev(0x562d1fb5a000 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2023-12-01T06:24:48.444 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.438+0000 7ffae66bb940 1 bdev(0x562d1fb5a000 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:24:48.444 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.438+0000 7ffae66bb940 1 bluestore(td/osd-mapper/2) _set_cache_sizes cache_size 1073741824 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2023-12-01T06:24:48.444 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.438+0000 7ffae66bb940 1 bdev(0x562d1fb5a700 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2023-12-01T06:24:48.444 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.438+0000 7ffae66bb940 1 bdev(0x562d1fb5a700 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:24:48.444 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.438+0000 7ffae66bb940 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2023-12-01T06:24:48.444 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.438+0000 7ffae66bb940 1 bluefs mount 2023-12-01T06:24:48.451 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.446+0000 7ffae66bb940 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2023-12-01T06:24:48.468 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.462+0000 7ffae66bb940 1 bluefs mount shared_bdev_used = 0 2023-12-01T06:24:48.468 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.462+0000 7ffae66bb940 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2023-12-01T06:24:48.478 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.470+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2023-12-01T06:24:48.479 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.470+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2023-12-01T06:24:48.479 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.474+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2023-12-01T06:24:48.479 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.474+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2023-12-01T06:24:48.479 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.474+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2023-12-01T06:24:48.479 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.474+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2023-12-01T06:24:48.479 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.474+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2023-12-01T06:24:48.479 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.474+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2023-12-01T06:24:48.479 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.474+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2023-12-01T06:24:48.480 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.474+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2023-12-01T06:24:48.480 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.474+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2023-12-01T06:24:48.480 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.474+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2023-12-01T06:24:48.673 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.666+0000 7ffae66bb940 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2023-12-01T06:24:48.673 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.666+0000 7ffae66bb940 1 bluestore(td/osd-mapper/2) _open_super_meta old nid_max 2050 2023-12-01T06:24:48.673 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.666+0000 7ffae66bb940 1 bluestore(td/osd-mapper/2) _open_super_meta old blobid_max 20480 2023-12-01T06:24:48.674 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.666+0000 7ffae66bb940 1 bluestore(td/osd-mapper/2) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2023-12-01T06:24:48.674 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.666+0000 7ffae66bb940 1 bluestore(td/osd-mapper/2) _open_super_meta min_alloc_size 0x1000 2023-12-01T06:24:48.674 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.670+0000 7ffae66bb940 1 freelist init 2023-12-01T06:24:48.674 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.670+0000 7ffae66bb940 1 freelist _read_cfg 2023-12-01T06:24:48.677 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.674+0000 7ffae66bb940 1 bluestore(td/osd-mapper/2) _init_alloc loaded 100 GiB in 14 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc0000, fragmentation 4.95912e-07 2023-12-01T06:24:48.678 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.674+0000 7ffae66bb940 1 bluefs umount 2023-12-01T06:24:48.678 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.674+0000 7ffae66bb940 1 bdev(0x562d1fb5a700 td/osd-mapper/2/block) close 2023-12-01T06:24:48.729 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.726+0000 7ffae66bb940 1 bdev(0x562d1fb5a700 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2023-12-01T06:24:48.730 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.726+0000 7ffae66bb940 1 bdev(0x562d1fb5a700 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:24:48.730 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.726+0000 7ffae66bb940 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2023-12-01T06:24:48.730 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.726+0000 7ffae66bb940 1 bluefs mount 2023-12-01T06:24:48.730 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.726+0000 7ffae66bb940 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2023-12-01T06:24:48.735 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.730+0000 7ffae66bb940 1 bluefs mount shared_bdev_used = 27459584 2023-12-01T06:24:48.736 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.730+0000 7ffae66bb940 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2023-12-01T06:24:48.744 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.738+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2023-12-01T06:24:48.744 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.738+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2023-12-01T06:24:48.744 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.738+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2023-12-01T06:24:48.744 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.738+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2023-12-01T06:24:48.744 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.738+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2023-12-01T06:24:48.744 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.738+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2023-12-01T06:24:48.744 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.738+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2023-12-01T06:24:48.745 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.738+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2023-12-01T06:24:48.745 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.738+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2023-12-01T06:24:48.745 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.738+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2023-12-01T06:24:48.745 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.738+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2023-12-01T06:24:48.745 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:48.738+0000 7ffae66bb940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2023-12-01T06:24:49.767 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:49.762+0000 7ffae66bb940 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2023-12-01T06:24:49.772 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:49.766+0000 7ffae66bb940 1 bluefs umount 2023-12-01T06:24:49.772 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:49.766+0000 7ffae66bb940 1 bdev(0x562d1fb5a700 td/osd-mapper/2/block) close 2023-12-01T06:24:50.025 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:50.022+0000 7ffae66bb940 1 freelist shutdown 2023-12-01T06:24:50.025 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:50.022+0000 7ffae66bb940 1 bdev(0x562d1fb5a000 td/osd-mapper/2/block) close 2023-12-01T06:24:50.242 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:24:50.243 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:24:50.243 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:24:50.243 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:24:50.243 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:24:50.245 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:101: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2023-12-01T06:24:50.245 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:101: TEST_truncated_sna_record: grep -a SNA_ /tmp/oo2.dump 2023-12-01T06:24:50.245 INFO:tasks.workunit.client.0.smithi170.stderr:grep: /tmp/oo2.dump: No such file or directory 2023-12-01T06:24:50.245 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:102: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2023-12-01T06:24:50.245 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:102: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2023-12-01T06:24:50.849 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2023-12-01T06:24:50.850 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 0b 05 58 02 00 00 d9 5d 2c c3 e2 b2 44 6d b9 31 |..X....],...Dm.1| 2023-12-01T06:24:50.850 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 8d 97 09 2c c0 17 02 00 00 00 1c 00 00 00 00 00 |...,............| 2023-12-01T06:24:50.850 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-12-01T06:24:50.850 INFO:tasks.workunit.client.0.smithi170.stdout:* 2023-12-01T06:24:50.850 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2023-12-01T06:24:50.850 INFO:tasks.workunit.client.0.smithi170.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2023-12-01T06:24:50.850 INFO:tasks.workunit.client.0.smithi170.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2023-12-01T06:24:50.851 INFO:tasks.workunit.client.0.smithi170.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2023-12-01T06:24:50.851 INFO:tasks.workunit.client.0.smithi170.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2023-12-01T06:24:50.851 INFO:tasks.workunit.client.0.smithi170.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2023-12-01T06:24:50.851 INFO:tasks.workunit.client.0.smithi170.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2023-12-01T06:24:50.851 INFO:tasks.workunit.client.0.smithi170.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2023-12-01T06:24:50.851 INFO:tasks.workunit.client.0.smithi170.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2023-12-01T06:24:50.851 INFO:tasks.workunit.client.0.smithi170.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2023-12-01T06:24:50.851 INFO:tasks.workunit.client.0.smithi170.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2023-12-01T06:24:50.851 INFO:tasks.workunit.client.0.smithi170.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2023-12-01T06:24:50.852 INFO:tasks.workunit.client.0.smithi170.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2023-12-01T06:24:50.852 INFO:tasks.workunit.client.0.smithi170.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2023-12-01T06:24:50.852 INFO:tasks.workunit.client.0.smithi170.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2023-12-01T06:24:50.852 INFO:tasks.workunit.client.0.smithi170.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2023-12-01T06:24:50.852 INFO:tasks.workunit.client.0.smithi170.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2023-12-01T06:24:50.852 INFO:tasks.workunit.client.0.smithi170.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2023-12-01T06:24:50.852 INFO:tasks.workunit.client.0.smithi170.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2023-12-01T06:24:50.853 INFO:tasks.workunit.client.0.smithi170.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2023-12-01T06:24:50.853 INFO:tasks.workunit.client.0.smithi170.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2023-12-01T06:24:50.853 INFO:tasks.workunit.client.0.smithi170.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2023-12-01T06:24:50.853 INFO:tasks.workunit.client.0.smithi170.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2023-12-01T06:24:50.853 INFO:tasks.workunit.client.0.smithi170.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2023-12-01T06:24:50.853 INFO:tasks.workunit.client.0.smithi170.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2023-12-01T06:24:50.853 INFO:tasks.workunit.client.0.smithi170.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2023-12-01T06:24:50.854 INFO:tasks.workunit.client.0.smithi170.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2023-12-01T06:24:50.854 INFO:tasks.workunit.client.0.smithi170.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2023-12-01T06:24:50.854 INFO:tasks.workunit.client.0.smithi170.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2023-12-01T06:24:50.854 INFO:tasks.workunit.client.0.smithi170.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2023-12-01T06:24:50.854 INFO:tasks.workunit.client.0.smithi170.stdout:00000220 72 65 1c 00 00 00 0f 00 00 00 9a 24 47 a8 77 da |re.........$G.w.| 2023-12-01T06:24:50.854 INFO:tasks.workunit.client.0.smithi170.stdout:00000230 4c dc 85 fd 36 7e bd 30 bf 9b 00 00 00 00 00 00 |L...6~.0........| 2023-12-01T06:24:50.854 INFO:tasks.workunit.client.0.smithi170.stdout:00000240 00 00 1c 00 00 00 18 7c 69 65 cb 62 93 2a 01 00 |.......|ie.b.*..| 2023-12-01T06:24:50.855 INFO:tasks.workunit.client.0.smithi170.stdout:00000250 00 00 01 00 00 00 01 00 00 00 1c 00 00 00 |..............| 2023-12-01T06:24:50.855 INFO:tasks.workunit.client.0.smithi170.stdout:0000025e 2023-12-01T06:24:50.855 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:50.855 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2023-12-01T06:24:50.855 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:50.855 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:24:50.855 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2023-12-01T06:24:50.855 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2023-12-01T06:24:50.855 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:24:50.855 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2023-12-01T06:24:50.855 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 00 |.......| 2023-12-01T06:24:50.855 INFO:tasks.workunit.client.0.smithi170.stdout:00000047 2023-12-01T06:24:50.855 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout:0000003f 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 00 |.......| 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout:00000047 2023-12-01T06:24:50.856 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:50.857 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:24:50.857 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:50.857 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:50.857 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:50.857 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:50.857 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:50.857 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:50.858 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:24:50.858 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:50.858 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:50.858 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:50.859 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:50.860 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:50.860 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:50.860 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:24:50.860 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:50.860 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:50.860 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:50.860 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:50.860 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:50.860 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:50.860 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2023-12-01T06:24:50.860 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:50.860 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2023-12-01T06:24:50.860 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:51.325 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2023-12-01T06:24:51.325 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_000000000000000[0-9]_000000000000000' 2023-12-01T06:24:51.325 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: awk -e '{print $2;}' 2023-12-01T06:24:51.325 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: wc -l 2023-12-01T06:24:53.150 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: local num_sna_b4=5 2023-12-01T06:24:53.151 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: expr 3 - 1 2023-12-01T06:24:53.153 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: seq 0 2 2023-12-01T06:24:53.154 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2023-12-01T06:24:53.155 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:108: TEST_truncated_sna_record: kvdir=td/osd-mapper/0 2023-12-01T06:24:53.155 INFO:tasks.workunit.client.0.smithi170.stdout:corrupting the SnapMapper DB of osd.0 (db: td/osd-mapper/0) 2023-12-01T06:24:53.155 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:109: TEST_truncated_sna_record: echo 'corrupting the SnapMapper DB of osd.0 (db: td/osd-mapper/0)' 2023-12-01T06:24:53.155 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: (( extr_dbg >= 3 )) 2023-12-01T06:24:53.155 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 dump p 2023-12-01T06:24:53.185 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.178+0000 7faea44f9940 1 bdev(0x55940bd1a000 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2023-12-01T06:24:53.185 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.182+0000 7faea44f9940 1 bdev(0x55940bd1a000 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:24:53.185 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.182+0000 7faea44f9940 1 bluestore(td/osd-mapper/0) _set_cache_sizes cache_size 1073741824 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2023-12-01T06:24:53.185 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.182+0000 7faea44f9940 1 bdev(0x55940bd1a700 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2023-12-01T06:24:53.185 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.182+0000 7faea44f9940 1 bdev(0x55940bd1a700 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:24:53.185 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.182+0000 7faea44f9940 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2023-12-01T06:24:53.186 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.182+0000 7faea44f9940 1 bluefs mount 2023-12-01T06:24:53.198 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.194+0000 7faea44f9940 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2023-12-01T06:24:53.208 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.202+0000 7faea44f9940 1 bluefs mount shared_bdev_used = 0 2023-12-01T06:24:53.208 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.202+0000 7faea44f9940 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2023-12-01T06:24:53.216 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.210+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2023-12-01T06:24:53.217 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.210+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2023-12-01T06:24:53.217 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.210+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2023-12-01T06:24:53.217 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.210+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2023-12-01T06:24:53.217 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.214+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2023-12-01T06:24:53.218 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.214+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2023-12-01T06:24:53.218 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.214+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2023-12-01T06:24:53.218 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.214+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2023-12-01T06:24:53.218 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.214+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2023-12-01T06:24:53.218 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.214+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2023-12-01T06:24:53.218 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.214+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2023-12-01T06:24:53.218 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.214+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2023-12-01T06:24:53.224 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.218+0000 7faea44f9940 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2023-12-01T06:24:53.225 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.218+0000 7faea44f9940 1 bluestore(td/osd-mapper/0) _open_super_meta old nid_max 2050 2023-12-01T06:24:53.225 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.218+0000 7faea44f9940 1 bluestore(td/osd-mapper/0) _open_super_meta old blobid_max 20480 2023-12-01T06:24:53.225 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.218+0000 7faea44f9940 1 bluestore(td/osd-mapper/0) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2023-12-01T06:24:53.225 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.218+0000 7faea44f9940 1 bluestore(td/osd-mapper/0) _open_super_meta min_alloc_size 0x1000 2023-12-01T06:24:53.225 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.218+0000 7faea44f9940 1 freelist init 2023-12-01T06:24:53.225 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.218+0000 7faea44f9940 1 freelist _read_cfg 2023-12-01T06:24:53.228 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.222+0000 7faea44f9940 1 bluestore(td/osd-mapper/0) _init_alloc loaded 100 GiB in 24 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc0000, fragmentation 8.77383e-07 2023-12-01T06:24:53.229 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.222+0000 7faea44f9940 1 bluefs umount 2023-12-01T06:24:53.229 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.222+0000 7faea44f9940 1 bdev(0x55940bd1a700 td/osd-mapper/0/block) close 2023-12-01T06:24:53.465 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.462+0000 7faea44f9940 1 bdev(0x55940bd1a700 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2023-12-01T06:24:53.465 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.462+0000 7faea44f9940 1 bdev(0x55940bd1a700 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:24:53.465 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.462+0000 7faea44f9940 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2023-12-01T06:24:53.465 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.462+0000 7faea44f9940 1 bluefs mount 2023-12-01T06:24:53.473 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.470+0000 7faea44f9940 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2023-12-01T06:24:53.483 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.478+0000 7faea44f9940 1 bluefs mount shared_bdev_used = 9568256 2023-12-01T06:24:53.484 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.478+0000 7faea44f9940 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2023-12-01T06:24:53.493 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.490+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2023-12-01T06:24:53.493 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.490+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2023-12-01T06:24:53.493 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.490+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2023-12-01T06:24:53.494 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.490+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2023-12-01T06:24:53.494 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.490+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2023-12-01T06:24:53.494 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.490+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2023-12-01T06:24:53.494 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.490+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2023-12-01T06:24:53.494 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.490+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2023-12-01T06:24:53.495 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.490+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2023-12-01T06:24:53.495 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.490+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2023-12-01T06:24:53.495 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.490+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2023-12-01T06:24:53.495 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.490+0000 7faea44f9940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2023-12-01T06:24:53.742 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.738+0000 7faea44f9940 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2023-12-01T06:24:53.746 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2023-12-01T06:24:53.746 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 0b 05 58 02 00 00 d9 5d 2c c3 e2 b2 44 6d b9 31 |..X....],...Dm.1| 2023-12-01T06:24:53.746 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 8d 97 09 2c c0 17 00 00 00 00 1c 00 00 00 00 00 |...,............| 2023-12-01T06:24:53.747 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-12-01T06:24:53.747 INFO:tasks.workunit.client.0.smithi170.stdout:* 2023-12-01T06:24:53.747 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2023-12-01T06:24:53.747 INFO:tasks.workunit.client.0.smithi170.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2023-12-01T06:24:53.747 INFO:tasks.workunit.client.0.smithi170.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2023-12-01T06:24:53.747 INFO:tasks.workunit.client.0.smithi170.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2023-12-01T06:24:53.747 INFO:tasks.workunit.client.0.smithi170.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2023-12-01T06:24:53.747 INFO:tasks.workunit.client.0.smithi170.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2023-12-01T06:24:53.747 INFO:tasks.workunit.client.0.smithi170.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2023-12-01T06:24:53.747 INFO:tasks.workunit.client.0.smithi170.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2023-12-01T06:24:53.747 INFO:tasks.workunit.client.0.smithi170.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2023-12-01T06:24:53.747 INFO:tasks.workunit.client.0.smithi170.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2023-12-01T06:24:53.747 INFO:tasks.workunit.client.0.smithi170.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2023-12-01T06:24:53.747 INFO:tasks.workunit.client.0.smithi170.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2023-12-01T06:24:53.748 INFO:tasks.workunit.client.0.smithi170.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout:00000220 72 65 1c 00 00 00 05 00 00 00 80 5e f5 92 ce 40 |re.........^...@| 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout:00000230 4f ba 84 bd bd 7d b4 bf d1 4d 00 00 00 00 00 00 |O....}...M......| 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout:00000240 00 00 1c 00 00 00 d5 7b 69 65 13 e6 cb 07 01 00 |.......{ie......| 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout:00000250 00 00 01 00 00 00 01 00 00 00 1c 00 00 00 |..............| 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout:0000025e 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:24:53.749 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 00 |.......| 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout:00000047 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout:0000003f 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:24:53.750 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 00 |.......| 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout:00000047 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:53.751 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:53.752 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:53.753 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:53.753 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:24:53.753 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:53.753 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:53.753 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:53.753 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:53.753 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:53.753 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:53.753 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2023-12-01T06:24:53.753 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:53.753 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2023-12-01T06:24:53.753 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:53.754 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.742+0000 7faea44f9940 1 bluefs umount 2023-12-01T06:24:53.754 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:53.742+0000 7faea44f9940 1 bdev(0x55940bd1a700 td/osd-mapper/0/block) close 2023-12-01T06:24:54.005 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:54.002+0000 7faea44f9940 1 freelist shutdown 2023-12-01T06:24:54.005 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:54.002+0000 7faea44f9940 1 bdev(0x55940bd1a000 td/osd-mapper/0/block) close 2023-12-01T06:24:54.235 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 dump p 2023-12-01T06:24:54.235 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_0000000000000003_000000000000000' 2023-12-01T06:24:54.235 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: awk -e '{print $2;}' 2023-12-01T06:24:55.308 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: KY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:24:55.309 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2023-12-01T06:24:55.309 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: echo 'SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..' 2023-12-01T06:24:55.309 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: cat -v 2023-12-01T06:24:55.310 INFO:tasks.workunit.client.0.smithi170.stdout:SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:24:55.311 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: mktemp -p /tmp --suffix=_the_val 2023-12-01T06:24:55.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: tmp_fn1=/tmp/tmp.oVKLpG0dGu_the_val 2023-12-01T06:24:55.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2023-12-01T06:24:55.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: echo 'Value dumped in: /tmp/tmp.oVKLpG0dGu_the_val' 2023-12-01T06:24:55.313 INFO:tasks.workunit.client.0.smithi170.stdout:Value dumped in: /tmp/tmp.oVKLpG0dGu_the_val 2023-12-01T06:24:55.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:119: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 get p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. out /tmp/tmp.oVKLpG0dGu_the_val 2023-12-01T06:24:55.889 INFO:tasks.workunit.client.0.smithi170.stdout:(p, %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..) 2023-12-01T06:24:56.383 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2023-12-01T06:24:56.383 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: od -xc /tmp/tmp.oVKLpG0dGu_the_val 2023-12-01T06:24:56.395 INFO:tasks.workunit.client.0.smithi170.stdout:0000000 0101 0035 0000 0003 0000 0000 0000 0304 2023-12-01T06:24:56.395 INFO:tasks.workunit.client.0.smithi170.stdout: 001 001 5 \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0 004 003 2023-12-01T06:24:56.395 INFO:tasks.workunit.client.0.smithi170.stdout:0000020 0027 0000 0000 0000 0006 0000 626f 786a 2023-12-01T06:24:56.395 INFO:tasks.workunit.client.0.smithi170.stdout: ' \0 \0 \0 \0 \0 \0 \0 006 \0 \0 \0 o b j x 2023-12-01T06:24:56.395 INFO:tasks.workunit.client.0.smithi170.stdout:0000040 7878 0003 0000 0000 0000 75af 85d0 0000 2023-12-01T06:24:56.395 INFO:tasks.workunit.client.0.smithi170.stdout: x x 003 \0 \0 \0 \0 \0 \0 \0 � u � 205 \0 \0 2023-12-01T06:24:56.395 INFO:tasks.workunit.client.0.smithi170.stdout:0000060 0000 0100 0000 0000 0000 0000 2023-12-01T06:24:56.395 INFO:tasks.workunit.client.0.smithi170.stdout: \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 2023-12-01T06:24:56.395 INFO:tasks.workunit.client.0.smithi170.stdout:0000073 2023-12-01T06:24:56.396 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:122: TEST_truncated_sna_record: NKY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2023-12-01T06:24:56.396 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:123: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 rm p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:24:57.463 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:124: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 set p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 in /tmp/tmp.oVKLpG0dGu_the_val 2023-12-01T06:24:58.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:126: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2023-12-01T06:24:58.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2023-12-01T06:24:58.792 INFO:tasks.workunit.client.0.smithi170.stdout:corrupting the SnapMapper DB of osd.1 (db: td/osd-mapper/1) 2023-12-01T06:24:58.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:108: TEST_truncated_sna_record: kvdir=td/osd-mapper/1 2023-12-01T06:24:58.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:109: TEST_truncated_sna_record: echo 'corrupting the SnapMapper DB of osd.1 (db: td/osd-mapper/1)' 2023-12-01T06:24:58.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: (( extr_dbg >= 3 )) 2023-12-01T06:24:58.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2023-12-01T06:24:58.821 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.814+0000 7f4d20668940 1 bdev(0x55ad37e62000 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2023-12-01T06:24:58.821 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.818+0000 7f4d20668940 1 bdev(0x55ad37e62000 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:24:58.821 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.818+0000 7f4d20668940 1 bluestore(td/osd-mapper/1) _set_cache_sizes cache_size 1073741824 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2023-12-01T06:24:58.821 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.818+0000 7f4d20668940 1 bdev(0x55ad37e62700 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2023-12-01T06:24:58.822 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.818+0000 7f4d20668940 1 bdev(0x55ad37e62700 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:24:58.822 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.818+0000 7f4d20668940 1 bluefs add_block_device bdev 1 path td/osd-mapper/1/block size 100 GiB 2023-12-01T06:24:58.822 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.818+0000 7f4d20668940 1 bluefs mount 2023-12-01T06:24:58.830 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.826+0000 7f4d20668940 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2023-12-01T06:24:58.842 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.838+0000 7f4d20668940 1 bluefs mount shared_bdev_used = 0 2023-12-01T06:24:58.842 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.838+0000 7f4d20668940 1 bluestore(td/osd-mapper/1) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2023-12-01T06:24:58.850 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.846+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2023-12-01T06:24:58.850 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.846+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2023-12-01T06:24:58.851 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.846+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2023-12-01T06:24:58.851 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.846+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2023-12-01T06:24:58.851 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.846+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2023-12-01T06:24:58.851 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.846+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2023-12-01T06:24:58.852 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.846+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2023-12-01T06:24:58.852 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.846+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2023-12-01T06:24:58.852 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.846+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2023-12-01T06:24:58.852 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.846+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2023-12-01T06:24:58.852 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.846+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2023-12-01T06:24:58.852 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.846+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2023-12-01T06:24:58.859 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.854+0000 7f4d20668940 1 bluestore(td/osd-mapper/1) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2023-12-01T06:24:58.859 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.854+0000 7f4d20668940 1 bluestore(td/osd-mapper/1) _open_super_meta old nid_max 2050 2023-12-01T06:24:58.859 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.854+0000 7f4d20668940 1 bluestore(td/osd-mapper/1) _open_super_meta old blobid_max 20480 2023-12-01T06:24:58.859 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.854+0000 7f4d20668940 1 bluestore(td/osd-mapper/1) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2023-12-01T06:24:58.859 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.854+0000 7f4d20668940 1 bluestore(td/osd-mapper/1) _open_super_meta min_alloc_size 0x1000 2023-12-01T06:24:58.859 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.854+0000 7f4d20668940 1 freelist init 2023-12-01T06:24:58.859 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.854+0000 7f4d20668940 1 freelist _read_cfg 2023-12-01T06:24:58.862 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.858+0000 7f4d20668940 1 bluestore(td/osd-mapper/1) _init_alloc loaded 100 GiB in 16 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc0000, fragmentation 5.72206e-07 2023-12-01T06:24:58.863 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.858+0000 7f4d20668940 1 bluefs umount 2023-12-01T06:24:58.863 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:58.858+0000 7f4d20668940 1 bdev(0x55ad37e62700 td/osd-mapper/1/block) close 2023-12-01T06:24:59.101 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.098+0000 7f4d20668940 1 bdev(0x55ad37e62700 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2023-12-01T06:24:59.101 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.098+0000 7f4d20668940 1 bdev(0x55ad37e62700 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:24:59.102 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.098+0000 7f4d20668940 1 bluefs add_block_device bdev 1 path td/osd-mapper/1/block size 100 GiB 2023-12-01T06:24:59.102 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.098+0000 7f4d20668940 1 bluefs mount 2023-12-01T06:24:59.106 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.102+0000 7f4d20668940 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2023-12-01T06:24:59.118 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.114+0000 7f4d20668940 1 bluefs mount shared_bdev_used = 9568256 2023-12-01T06:24:59.118 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.114+0000 7f4d20668940 1 bluestore(td/osd-mapper/1) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2023-12-01T06:24:59.132 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.126+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2023-12-01T06:24:59.132 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.126+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2023-12-01T06:24:59.133 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.130+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2023-12-01T06:24:59.133 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.130+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2023-12-01T06:24:59.133 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.130+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2023-12-01T06:24:59.133 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.130+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2023-12-01T06:24:59.134 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.130+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2023-12-01T06:24:59.134 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.130+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2023-12-01T06:24:59.134 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.130+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2023-12-01T06:24:59.135 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.130+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2023-12-01T06:24:59.135 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.130+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2023-12-01T06:24:59.135 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.130+0000 7f4d20668940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2023-12-01T06:24:59.501 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.498+0000 7f4d20668940 1 bluestore(td/osd-mapper/1) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2023-12-01T06:24:59.504 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2023-12-01T06:24:59.504 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 0b 05 58 02 00 00 d9 5d 2c c3 e2 b2 44 6d b9 31 |..X....],...Dm.1| 2023-12-01T06:24:59.504 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 8d 97 09 2c c0 17 01 00 00 00 1c 00 00 00 00 00 |...,............| 2023-12-01T06:24:59.504 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-12-01T06:24:59.504 INFO:tasks.workunit.client.0.smithi170.stdout:* 2023-12-01T06:24:59.504 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2023-12-01T06:24:59.504 INFO:tasks.workunit.client.0.smithi170.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2023-12-01T06:24:59.504 INFO:tasks.workunit.client.0.smithi170.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2023-12-01T06:24:59.505 INFO:tasks.workunit.client.0.smithi170.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:00000220 72 65 1c 00 00 00 0a 00 00 00 28 54 28 83 7e 5e |re........(T(.~^| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:00000230 4a 91 98 30 a7 34 fb 6a d0 ff 00 00 00 00 00 00 |J..0.4.j........| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:00000240 00 00 1c 00 00 00 f6 7b 69 65 cc 5c 4d 25 01 00 |.......{ie.\M%..| 2023-12-01T06:24:59.506 INFO:tasks.workunit.client.0.smithi170.stdout:00000250 00 00 01 00 00 00 01 00 00 00 1c 00 00 00 |..............| 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout:0000025e 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 00 |.......| 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout:00000047 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2023-12-01T06:24:59.507 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout:0000003f 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 00 |.......| 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout:00000047 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:59.508 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:59.509 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:24:59.510 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:59.511 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2023-12-01T06:24:59.511 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:59.511 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2023-12-01T06:24:59.511 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:24:59.511 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.502+0000 7f4d20668940 1 bluefs umount 2023-12-01T06:24:59.511 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.502+0000 7f4d20668940 1 bdev(0x55ad37e62700 td/osd-mapper/1/block) close 2023-12-01T06:24:59.757 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.754+0000 7f4d20668940 1 freelist shutdown 2023-12-01T06:24:59.757 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:24:59.754+0000 7f4d20668940 1 bdev(0x55ad37e62000 td/osd-mapper/1/block) close 2023-12-01T06:24:59.867 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2023-12-01T06:24:59.867 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_0000000000000003_000000000000000' 2023-12-01T06:24:59.867 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: awk -e '{print $2;}' 2023-12-01T06:25:00.936 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: KY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:25:00.936 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2023-12-01T06:25:00.937 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: echo 'SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..' 2023-12-01T06:25:00.937 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: cat -v 2023-12-01T06:25:00.938 INFO:tasks.workunit.client.0.smithi170.stdout:SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:25:00.939 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: mktemp -p /tmp --suffix=_the_val 2023-12-01T06:25:00.941 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: tmp_fn1=/tmp/tmp.gq7xwefnXe_the_val 2023-12-01T06:25:00.941 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2023-12-01T06:25:00.941 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: echo 'Value dumped in: /tmp/tmp.gq7xwefnXe_the_val' 2023-12-01T06:25:00.941 INFO:tasks.workunit.client.0.smithi170.stdout:Value dumped in: /tmp/tmp.gq7xwefnXe_the_val 2023-12-01T06:25:00.941 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:119: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 get p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. out /tmp/tmp.gq7xwefnXe_the_val 2023-12-01T06:25:01.597 INFO:tasks.workunit.client.0.smithi170.stdout:(p, %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..) 2023-12-01T06:25:02.011 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2023-12-01T06:25:02.011 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: od -xc /tmp/tmp.gq7xwefnXe_the_val 2023-12-01T06:25:02.012 INFO:tasks.workunit.client.0.smithi170.stdout:0000000 0101 0035 0000 0003 0000 0000 0000 0304 2023-12-01T06:25:02.012 INFO:tasks.workunit.client.0.smithi170.stdout: 001 001 5 \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0 004 003 2023-12-01T06:25:02.013 INFO:tasks.workunit.client.0.smithi170.stdout:0000020 0027 0000 0000 0000 0006 0000 626f 786a 2023-12-01T06:25:02.013 INFO:tasks.workunit.client.0.smithi170.stdout: ' \0 \0 \0 \0 \0 \0 \0 006 \0 \0 \0 o b j x 2023-12-01T06:25:02.013 INFO:tasks.workunit.client.0.smithi170.stdout:0000040 7878 0003 0000 0000 0000 75af 85d0 0000 2023-12-01T06:25:02.013 INFO:tasks.workunit.client.0.smithi170.stdout: x x 003 \0 \0 \0 \0 \0 \0 \0 � u � 205 \0 \0 2023-12-01T06:25:02.013 INFO:tasks.workunit.client.0.smithi170.stdout:0000060 0000 0100 0000 0000 0000 0000 2023-12-01T06:25:02.013 INFO:tasks.workunit.client.0.smithi170.stdout: \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 2023-12-01T06:25:02.013 INFO:tasks.workunit.client.0.smithi170.stdout:0000073 2023-12-01T06:25:02.013 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:122: TEST_truncated_sna_record: NKY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2023-12-01T06:25:02.013 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:123: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 rm p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:25:03.083 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:124: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 set p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 in /tmp/tmp.gq7xwefnXe_the_val 2023-12-01T06:25:04.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:126: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2023-12-01T06:25:04.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2023-12-01T06:25:04.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:108: TEST_truncated_sna_record: kvdir=td/osd-mapper/2 2023-12-01T06:25:04.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:109: TEST_truncated_sna_record: echo 'corrupting the SnapMapper DB of osd.2 (db: td/osd-mapper/2)' 2023-12-01T06:25:04.404 INFO:tasks.workunit.client.0.smithi170.stdout:corrupting the SnapMapper DB of osd.2 (db: td/osd-mapper/2) 2023-12-01T06:25:04.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: (( extr_dbg >= 3 )) 2023-12-01T06:25:04.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2023-12-01T06:25:04.432 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.426+0000 7f851841a940 1 bdev(0x55c3a880e000 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2023-12-01T06:25:04.433 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.426+0000 7f851841a940 1 bdev(0x55c3a880e000 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:25:04.433 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.426+0000 7f851841a940 1 bluestore(td/osd-mapper/2) _set_cache_sizes cache_size 1073741824 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2023-12-01T06:25:04.433 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.426+0000 7f851841a940 1 bdev(0x55c3a880e700 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2023-12-01T06:25:04.433 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.426+0000 7f851841a940 1 bdev(0x55c3a880e700 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:25:04.433 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.426+0000 7f851841a940 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2023-12-01T06:25:04.433 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.426+0000 7f851841a940 1 bluefs mount 2023-12-01T06:25:04.448 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.442+0000 7f851841a940 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2023-12-01T06:25:04.466 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.462+0000 7f851841a940 1 bluefs mount shared_bdev_used = 0 2023-12-01T06:25:04.466 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.462+0000 7f851841a940 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2023-12-01T06:25:04.475 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.470+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2023-12-01T06:25:04.475 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.470+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2023-12-01T06:25:04.476 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.470+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2023-12-01T06:25:04.476 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.470+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2023-12-01T06:25:04.476 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.470+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2023-12-01T06:25:04.476 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.470+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2023-12-01T06:25:04.477 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.470+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2023-12-01T06:25:04.477 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.470+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2023-12-01T06:25:04.477 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.474+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2023-12-01T06:25:04.477 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.474+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2023-12-01T06:25:04.477 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.474+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2023-12-01T06:25:04.477 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.474+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2023-12-01T06:25:04.484 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.478+0000 7f851841a940 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2023-12-01T06:25:04.484 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.478+0000 7f851841a940 1 bluestore(td/osd-mapper/2) _open_super_meta old nid_max 2050 2023-12-01T06:25:04.484 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.478+0000 7f851841a940 1 bluestore(td/osd-mapper/2) _open_super_meta old blobid_max 20480 2023-12-01T06:25:04.485 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.478+0000 7f851841a940 1 bluestore(td/osd-mapper/2) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2023-12-01T06:25:04.485 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.478+0000 7f851841a940 1 bluestore(td/osd-mapper/2) _open_super_meta min_alloc_size 0x1000 2023-12-01T06:25:04.485 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.482+0000 7f851841a940 1 freelist init 2023-12-01T06:25:04.485 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.482+0000 7f851841a940 1 freelist _read_cfg 2023-12-01T06:25:04.488 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.486+0000 7f851841a940 1 bluestore(td/osd-mapper/2) _init_alloc loaded 100 GiB in 14 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc0000, fragmentation 4.95912e-07 2023-12-01T06:25:04.489 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.486+0000 7f851841a940 1 bluefs umount 2023-12-01T06:25:04.489 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.486+0000 7f851841a940 1 bdev(0x55c3a880e700 td/osd-mapper/2/block) close 2023-12-01T06:25:04.713 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.710+0000 7f851841a940 1 bdev(0x55c3a880e700 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2023-12-01T06:25:04.713 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.710+0000 7f851841a940 1 bdev(0x55c3a880e700 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) rotational device, discard not supported 2023-12-01T06:25:04.713 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.710+0000 7f851841a940 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2023-12-01T06:25:04.713 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.710+0000 7f851841a940 1 bluefs mount 2023-12-01T06:25:04.723 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.718+0000 7f851841a940 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2023-12-01T06:25:04.733 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.730+0000 7f851841a940 1 bluefs mount shared_bdev_used = 9568256 2023-12-01T06:25:04.733 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.730+0000 7f851841a940 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2023-12-01T06:25:04.743 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.738+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2023-12-01T06:25:04.743 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.738+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2023-12-01T06:25:04.743 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.738+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2023-12-01T06:25:04.744 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.738+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2023-12-01T06:25:04.745 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.738+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2023-12-01T06:25:04.745 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.738+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2023-12-01T06:25:04.745 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.738+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2023-12-01T06:25:04.745 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.738+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2023-12-01T06:25:04.745 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.742+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2023-12-01T06:25:04.745 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.742+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2023-12-01T06:25:04.745 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.742+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2023-12-01T06:25:04.745 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.742+0000 7f851841a940 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2023-12-01T06:25:04.943 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.938+0000 7f851841a940 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2023-12-01T06:25:04.946 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2023-12-01T06:25:04.946 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 0b 05 58 02 00 00 d9 5d 2c c3 e2 b2 44 6d b9 31 |..X....],...Dm.1| 2023-12-01T06:25:04.946 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 8d 97 09 2c c0 17 02 00 00 00 1c 00 00 00 00 00 |...,............| 2023-12-01T06:25:04.946 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-12-01T06:25:04.946 INFO:tasks.workunit.client.0.smithi170.stdout:* 2023-12-01T06:25:04.946 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2023-12-01T06:25:04.946 INFO:tasks.workunit.client.0.smithi170.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2023-12-01T06:25:04.946 INFO:tasks.workunit.client.0.smithi170.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2023-12-01T06:25:04.946 INFO:tasks.workunit.client.0.smithi170.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2023-12-01T06:25:04.946 INFO:tasks.workunit.client.0.smithi170.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2023-12-01T06:25:04.946 INFO:tasks.workunit.client.0.smithi170.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2023-12-01T06:25:04.947 INFO:tasks.workunit.client.0.smithi170.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout:00000220 72 65 1c 00 00 00 0f 00 00 00 9a 24 47 a8 77 da |re.........$G.w.| 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout:00000230 4c dc 85 fd 36 7e bd 30 bf 9b 00 00 00 00 00 00 |L...6~.0........| 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout:00000240 00 00 1c 00 00 00 18 7c 69 65 cb 62 93 2a 01 00 |.......|ie.b.*..| 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout:00000250 00 00 01 00 00 00 01 00 00 00 1c 00 00 00 |..............| 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout:0000025e 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:04.948 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 00 |.......| 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout:00000047 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout:0000003f 2023-12-01T06:25:04.949 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:04.950 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:25:04.950 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2023-12-01T06:25:04.950 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2023-12-01T06:25:04.950 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2023-12-01T06:25:04.950 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2023-12-01T06:25:04.950 INFO:tasks.workunit.client.0.smithi170.stdout:00000040 00 00 00 00 00 00 00 |.......| 2023-12-01T06:25:04.950 INFO:tasks.workunit.client.0.smithi170.stdout:00000047 2023-12-01T06:25:04.950 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:04.951 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2023-12-01T06:25:04.952 INFO:tasks.workunit.client.0.smithi170.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2023-12-01T06:25:04.953 INFO:tasks.workunit.client.0.smithi170.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2023-12-01T06:25:04.953 INFO:tasks.workunit.client.0.smithi170.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2023-12-01T06:25:04.953 INFO:tasks.workunit.client.0.smithi170.stdout:0000003b 2023-12-01T06:25:04.953 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:04.953 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2023-12-01T06:25:04.953 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:04.953 INFO:tasks.workunit.client.0.smithi170.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2023-12-01T06:25:04.953 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:04.954 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.942+0000 7f851841a940 1 bluefs umount 2023-12-01T06:25:04.954 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:04.942+0000 7f851841a940 1 bdev(0x55c3a880e700 td/osd-mapper/2/block) close 2023-12-01T06:25:05.205 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:05.202+0000 7f851841a940 1 freelist shutdown 2023-12-01T06:25:05.205 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:05.202+0000 7f851841a940 1 bdev(0x55c3a880e000 td/osd-mapper/2/block) close 2023-12-01T06:25:05.473 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2023-12-01T06:25:05.473 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_0000000000000003_000000000000000' 2023-12-01T06:25:05.473 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: awk -e '{print $2;}' 2023-12-01T06:25:06.544 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: KY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:25:06.545 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2023-12-01T06:25:06.545 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: echo 'SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..' 2023-12-01T06:25:06.545 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: cat -v 2023-12-01T06:25:06.545 INFO:tasks.workunit.client.0.smithi170.stdout:SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:25:06.546 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: mktemp -p /tmp --suffix=_the_val 2023-12-01T06:25:06.548 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: tmp_fn1=/tmp/tmp.Zcs8EQMN2i_the_val 2023-12-01T06:25:06.548 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2023-12-01T06:25:06.548 INFO:tasks.workunit.client.0.smithi170.stdout:Value dumped in: /tmp/tmp.Zcs8EQMN2i_the_val 2023-12-01T06:25:06.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: echo 'Value dumped in: /tmp/tmp.Zcs8EQMN2i_the_val' 2023-12-01T06:25:06.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:119: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 get p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. out /tmp/tmp.Zcs8EQMN2i_the_val 2023-12-01T06:25:07.104 INFO:tasks.workunit.client.0.smithi170.stdout:(p, %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..) 2023-12-01T06:25:07.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2023-12-01T06:25:07.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: od -xc /tmp/tmp.Zcs8EQMN2i_the_val 2023-12-01T06:25:07.621 INFO:tasks.workunit.client.0.smithi170.stdout:0000000 0101 0035 0000 0003 0000 0000 0000 0304 2023-12-01T06:25:07.621 INFO:tasks.workunit.client.0.smithi170.stdout: 001 001 5 \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0 004 003 2023-12-01T06:25:07.621 INFO:tasks.workunit.client.0.smithi170.stdout:0000020 0027 0000 0000 0000 0006 0000 626f 786a 2023-12-01T06:25:07.621 INFO:tasks.workunit.client.0.smithi170.stdout: ' \0 \0 \0 \0 \0 \0 \0 006 \0 \0 \0 o b j x 2023-12-01T06:25:07.621 INFO:tasks.workunit.client.0.smithi170.stdout:0000040 7878 0003 0000 0000 0000 75af 85d0 0000 2023-12-01T06:25:07.621 INFO:tasks.workunit.client.0.smithi170.stdout: x x 003 \0 \0 \0 \0 \0 \0 \0 � u � 205 \0 \0 2023-12-01T06:25:07.621 INFO:tasks.workunit.client.0.smithi170.stdout:0000060 0000 0100 0000 0000 0000 0000 2023-12-01T06:25:07.621 INFO:tasks.workunit.client.0.smithi170.stdout: \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 2023-12-01T06:25:07.621 INFO:tasks.workunit.client.0.smithi170.stdout:0000073 2023-12-01T06:25:07.622 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:122: TEST_truncated_sna_record: NKY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2023-12-01T06:25:07.622 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:123: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 rm p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:25:08.691 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:124: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 set p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 in /tmp/tmp.Zcs8EQMN2i_the_val 2023-12-01T06:25:10.011 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:126: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2023-12-01T06:25:10.011 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:129: TEST_truncated_sna_record: orig_osd_args=' --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1 ' 2023-12-01T06:25:10.011 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:130: TEST_truncated_sna_record: echo --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1 2023-12-01T06:25:10.012 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:130: TEST_truncated_sna_record: orig_osd_args=' --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1' 2023-12-01T06:25:10.012 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:131: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2023-12-01T06:25:10.012 INFO:tasks.workunit.client.0.smithi170.stdout:Copied OSD args: / --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1/ /--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1/ 2023-12-01T06:25:10.012 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:131: TEST_truncated_sna_record: echo 'Copied OSD args: / --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1/ /--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1/' 2023-12-01T06:25:10.013 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: expr 3 - 1 2023-12-01T06:25:10.015 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: seq 0 2 2023-12-01T06:25:10.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2023-12-01T06:25:10.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: CEPH_ARGS='--fsid=d95d2cc3-e2b2-446d-b931-8d97092cc017 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1' 2023-12-01T06:25:10.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: activate_osd td/osd-mapper 0 2023-12-01T06:25:10.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-mapper 2023-12-01T06:25:10.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2023-12-01T06:25:10.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2023-12-01T06:25:10.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2023-12-01T06:25:10.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-mapper/0 2023-12-01T06:25:10.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=d95d2cc3-e2b2-446d-b931-8d97092cc017 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1' 2023-12-01T06:25:10.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:25:10.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:25:10.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:25:10.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-mapper/0' 2023-12-01T06:25:10.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-mapper/0/journal' 2023-12-01T06:25:10.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2023-12-01T06:25:10.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2023-12-01T06:25:10.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-mapper' 2023-12-01T06:25:10.019 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2023-12-01T06:25:10.019 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:25:10.019 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:25:10.019 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:25:10.019 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:25:10.020 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.30922 2023-12-01T06:25:10.020 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.30922/$cluster-$name.asok' 2023-12-01T06:25:10.021 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.30922/$cluster-$name.asok' 2023-12-01T06:25:10.021 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:25:10.021 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-mapper/$name.log' 2023-12-01T06:25:10.021 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-mapper/$name.pid' 2023-12-01T06:25:10.022 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:25:10.022 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:25:10.022 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:25:10.022 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:25:10.022 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2023-12-01T06:25:10.022 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2023-12-01T06:25:10.022 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-mapper/0 2023-12-01T06:25:10.022 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2023-12-01T06:25:10.023 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.0 2023-12-01T06:25:10.024 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=d95d2cc3-e2b2-446d-b931-8d97092cc017 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-mapper/0 --osd-journal=td/osd-mapper/0/journal --chdir= --run-dir=td/osd-mapper '--admin-socket=/tmp/ceph-asok.30922/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-mapper/$name.log' '--pid-file=td/osd-mapper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2023-12-01T06:25:10.024 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-mapper/0/whoami 2023-12-01T06:25:10.024 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2023-12-01T06:25:10.024 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2023-12-01T06:25:10.024 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2023-12-01T06:25:10.025 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2023-12-01T06:25:10.062 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:10.058+0000 7f660af62640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:10.064 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:10.062+0000 7f660af62640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:10.066 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:10.062+0000 7f660af62640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:10.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2023-12-01T06:25:10.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:25:10.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2023-12-01T06:25:10.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:25:10.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:25:10.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:10.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:25:10.340 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:25:10.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:10.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:25:10.599 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:10.594+0000 7f660af62640 -1 Falling back to public interface 2023-12-01T06:25:10.714 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:25:11.716 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:25:11.716 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:11.716 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:25:11.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:25:11.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:11.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:25:12.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:25:13.039 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:13.034+0000 7f660af62640 -1 osd.0 28 log_to_monitors true 2023-12-01T06:25:13.043 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:25:13.043 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:13.043 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:25:13.044 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:25:13.044 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:13.044 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:25:13.472 INFO:tasks.workunit.client.0.smithi170.stdout:osd.0 up in weight 1 up_from 31 up_thru 17 down_at 29 last_clean_interval [5,28) [v2:127.0.0.1:6800/2226814035,v1:127.0.0.1:6801/2226814035] [v2:127.0.0.1:6802/2226814035,v1:127.0.0.1:6803/2226814035] exists,up 805ef592-ce40-4fba-84bd-bd7db4bfd14d 2023-12-01T06:25:13.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:25:13.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:25:13.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:25:13.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2023-12-01T06:25:13.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: CEPH_ARGS='--fsid=d95d2cc3-e2b2-446d-b931-8d97092cc017 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1' 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: activate_osd td/osd-mapper 1 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-mapper 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-mapper/1 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=d95d2cc3-e2b2-446d-b931-8d97092cc017 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1' 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-mapper/1' 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-mapper/1/journal' 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-mapper' 2023-12-01T06:25:13.475 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2023-12-01T06:25:13.476 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:25:13.476 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:25:13.476 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:25:13.476 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:25:13.476 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.30922 2023-12-01T06:25:13.476 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.30922/$cluster-$name.asok' 2023-12-01T06:25:13.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.30922/$cluster-$name.asok' 2023-12-01T06:25:13.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:25:13.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-mapper/$name.log' 2023-12-01T06:25:13.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-mapper/$name.pid' 2023-12-01T06:25:13.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:25:13.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:25:13.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:25:13.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:25:13.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2023-12-01T06:25:13.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2023-12-01T06:25:13.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-mapper/1 2023-12-01T06:25:13.478 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.1 2023-12-01T06:25:13.479 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2023-12-01T06:25:13.479 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=d95d2cc3-e2b2-446d-b931-8d97092cc017 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-mapper/1 --osd-journal=td/osd-mapper/1/journal --chdir= --run-dir=td/osd-mapper '--admin-socket=/tmp/ceph-asok.30922/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-mapper/$name.log' '--pid-file=td/osd-mapper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2023-12-01T06:25:13.479 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-mapper/1/whoami 2023-12-01T06:25:13.479 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2023-12-01T06:25:13.479 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2023-12-01T06:25:13.479 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2023-12-01T06:25:13.479 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2023-12-01T06:25:13.515 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:13.510+0000 7fa0f5d2a640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:13.531 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:13.526+0000 7fa0f5d2a640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:13.533 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:13.530+0000 7fa0f5d2a640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:13.801 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2023-12-01T06:25:13.801 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:25:13.801 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2023-12-01T06:25:13.801 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:25:13.801 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:25:13.802 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:25:13.802 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:13.802 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:25:13.802 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:13.802 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:25:14.067 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:14.062+0000 7fa0f5d2a640 -1 Falling back to public interface 2023-12-01T06:25:14.120 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:25:15.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:25:15.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:15.122 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:25:15.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:25:15.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:15.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:25:15.450 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:25:16.452 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:25:16.453 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:16.453 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:25:16.453 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:25:16.453 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:16.453 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:25:16.594 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:16.586+0000 7fa0f5d2a640 -1 osd.1 28 log_to_monitors true 2023-12-01T06:25:16.786 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:25:17.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:25:17.788 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:25:17.788 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:17.788 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:25:17.789 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:17.789 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:25:18.123 INFO:tasks.workunit.client.0.smithi170.stdout:osd.1 up in weight 1 up_from 34 up_thru 34 down_at 29 last_clean_interval [10,28) [v2:127.0.0.1:6810/1113679305,v1:127.0.0.1:6811/1113679305] [v2:127.0.0.1:6812/1113679305,v1:127.0.0.1:6813/1113679305] exists,up 28542883-7e5e-4a91-9830-a734fb6ad0ff 2023-12-01T06:25:18.127 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:25:18.127 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:25:18.127 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:25:18.127 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2023-12-01T06:25:18.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: CEPH_ARGS='--fsid=d95d2cc3-e2b2-446d-b931-8d97092cc017 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1' 2023-12-01T06:25:18.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: activate_osd td/osd-mapper 2 2023-12-01T06:25:18.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-mapper 2023-12-01T06:25:18.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2023-12-01T06:25:18.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2023-12-01T06:25:18.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2023-12-01T06:25:18.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-mapper/2 2023-12-01T06:25:18.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=d95d2cc3-e2b2-446d-b931-8d97092cc017 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1' 2023-12-01T06:25:18.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:25:18.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:25:18.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:25:18.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-mapper/2' 2023-12-01T06:25:18.129 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-mapper/2/journal' 2023-12-01T06:25:18.129 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2023-12-01T06:25:18.129 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2023-12-01T06:25:18.129 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-mapper' 2023-12-01T06:25:18.129 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2023-12-01T06:25:18.129 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:25:18.129 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:25:18.129 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:25:18.129 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:25:18.129 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.30922 2023-12-01T06:25:18.129 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.30922/$cluster-$name.asok' 2023-12-01T06:25:18.129 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.30922/$cluster-$name.asok' 2023-12-01T06:25:18.129 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:25:18.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-mapper/$name.log' 2023-12-01T06:25:18.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-mapper/$name.pid' 2023-12-01T06:25:18.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:25:18.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:25:18.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:25:18.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:25:18.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2023-12-01T06:25:18.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2023-12-01T06:25:18.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-mapper/2 2023-12-01T06:25:18.130 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.2 2023-12-01T06:25:18.131 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2023-12-01T06:25:18.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=d95d2cc3-e2b2-446d-b931-8d97092cc017 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-mapper/2 --osd-journal=td/osd-mapper/2/journal --chdir= --run-dir=td/osd-mapper '--admin-socket=/tmp/ceph-asok.30922/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-mapper/$name.log' '--pid-file=td/osd-mapper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2023-12-01T06:25:18.132 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-mapper/2/whoami 2023-12-01T06:25:18.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2023-12-01T06:25:18.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2023-12-01T06:25:18.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2023-12-01T06:25:18.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2023-12-01T06:25:18.175 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:18.170+0000 7fcb3fbd4640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:18.182 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:18.178+0000 7fcb3fbd4640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:18.183 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:18.178+0000 7fcb3fbd4640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:18.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2023-12-01T06:25:18.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:25:18.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2023-12-01T06:25:18.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:25:18.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:25:18.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:18.432 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:25:18.433 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:25:18.433 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:18.433 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:25:18.719 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:18.714+0000 7fcb3fbd4640 -1 Falling back to public interface 2023-12-01T06:25:18.740 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:25:19.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:25:19.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:19.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:25:19.742 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:25:19.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:19.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:25:20.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:25:21.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:25:21.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:21.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:25:21.070 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:25:21.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:21.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:25:21.163 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:21.158+0000 7fcb3fbd4640 -1 osd.2 28 log_to_monitors true 2023-12-01T06:25:21.412 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:25:22.413 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:25:22.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:22.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:25:22.414 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:25:22.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:22.415 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:25:22.856 INFO:tasks.workunit.client.0.smithi170.stdout:osd.2 up in weight 1 up_from 37 up_thru 37 down_at 29 last_clean_interval [15,28) [v2:127.0.0.1:6818/1981843167,v1:127.0.0.1:6819/1981843167] [v2:127.0.0.1:6820/1981843167,v1:127.0.0.1:6821/1981843167] exists,up 9a2447a8-77da-4cdc-85fd-367ebd30bf9b 2023-12-01T06:25:22.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:25:22.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:25:22.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:25:22.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:136: TEST_truncated_sna_record: sleep 1 2023-12-01T06:25:23.860 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: expr 3 - 1 2023-12-01T06:25:23.861 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: seq 0 2 2023-12-01T06:25:23.862 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2023-12-01T06:25:23.862 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:140: TEST_truncated_sna_record: timeout 60 ceph tell osd.0 version 2023-12-01T06:25:24.003 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:25:24.003 INFO:tasks.workunit.client.0.smithi170.stdout: "version": "18.0.0-7432-gbbeb098e", 2023-12-01T06:25:24.003 INFO:tasks.workunit.client.0.smithi170.stdout: "release": "reef", 2023-12-01T06:25:24.003 INFO:tasks.workunit.client.0.smithi170.stdout: "release_type": "dev" 2023-12-01T06:25:24.004 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:25:24.017 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2023-12-01T06:25:24.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:140: TEST_truncated_sna_record: timeout 60 ceph tell osd.1 version 2023-12-01T06:25:24.146 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:25:24.146 INFO:tasks.workunit.client.0.smithi170.stdout: "version": "18.0.0-7432-gbbeb098e", 2023-12-01T06:25:24.146 INFO:tasks.workunit.client.0.smithi170.stdout: "release": "reef", 2023-12-01T06:25:24.146 INFO:tasks.workunit.client.0.smithi170.stdout: "release_type": "dev" 2023-12-01T06:25:24.146 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:25:24.160 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2023-12-01T06:25:24.160 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:140: TEST_truncated_sna_record: timeout 60 ceph tell osd.2 version 2023-12-01T06:25:24.288 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:25:24.288 INFO:tasks.workunit.client.0.smithi170.stdout: "version": "18.0.0-7432-gbbeb098e", 2023-12-01T06:25:24.288 INFO:tasks.workunit.client.0.smithi170.stdout: "release": "reef", 2023-12-01T06:25:24.289 INFO:tasks.workunit.client.0.smithi170.stdout: "release_type": "dev" 2023-12-01T06:25:24.289 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:25:24.302 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:142: TEST_truncated_sna_record: rados --format json-pretty -p test listsnaps objxxx 2023-12-01T06:25:24.330 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: "name": "objxxx", 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: "clones": [ 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 2, 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: "snapshots": [ 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 1, 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: "name": "snap01" 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: }, 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 2, 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: "name": "snap02" 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: } 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: ], 2023-12-01T06:25:24.331 INFO:tasks.workunit.client.0.smithi170.stdout: "size": 6, 2023-12-01T06:25:24.332 INFO:tasks.workunit.client.0.smithi170.stdout: "overlaps": [] 2023-12-01T06:25:24.332 INFO:tasks.workunit.client.0.smithi170.stdout: }, 2023-12-01T06:25:24.332 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:25:24.332 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 3, 2023-12-01T06:25:24.332 INFO:tasks.workunit.client.0.smithi170.stdout: "snapshots": [ 2023-12-01T06:25:24.332 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:25:24.332 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 3, 2023-12-01T06:25:24.332 INFO:tasks.workunit.client.0.smithi170.stdout: "name": "snap13" 2023-12-01T06:25:24.332 INFO:tasks.workunit.client.0.smithi170.stdout: } 2023-12-01T06:25:24.332 INFO:tasks.workunit.client.0.smithi170.stdout: ], 2023-12-01T06:25:24.332 INFO:tasks.workunit.client.0.smithi170.stdout: "size": 6, 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: "overlaps": [] 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: }, 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 5, 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: "snapshots": [ 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 4, 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: "name": "snap24" 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: }, 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: "id": 5, 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: "name": "snap25" 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: } 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: ], 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: "size": 6, 2023-12-01T06:25:24.333 INFO:tasks.workunit.client.0.smithi170.stdout: "overlaps": [] 2023-12-01T06:25:24.334 INFO:tasks.workunit.client.0.smithi170.stdout: }, 2023-12-01T06:25:24.334 INFO:tasks.workunit.client.0.smithi170.stdout: { 2023-12-01T06:25:24.334 INFO:tasks.workunit.client.0.smithi170.stdout: "id": "head", 2023-12-01T06:25:24.334 INFO:tasks.workunit.client.0.smithi170.stdout: "snapshots": [], 2023-12-01T06:25:24.334 INFO:tasks.workunit.client.0.smithi170.stdout: "size": 6 2023-12-01T06:25:24.334 INFO:tasks.workunit.client.0.smithi170.stdout: } 2023-12-01T06:25:24.334 INFO:tasks.workunit.client.0.smithi170.stdout: ] 2023-12-01T06:25:24.334 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:25:24.334 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:145: TEST_truncated_sna_record: ceph osd unset nodeep-scrub 2023-12-01T06:25:24.717 INFO:tasks.workunit.client.0.smithi170.stderr:nodeep-scrub is unset 2023-12-01T06:25:24.738 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:146: TEST_truncated_sna_record: ceph osd unset noscrub 2023-12-01T06:25:25.161 INFO:tasks.workunit.client.0.smithi170.stderr:noscrub is unset 2023-12-01T06:25:25.183 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:149: TEST_truncated_sna_record: ceph --format=json-pretty osd map test objxxx 2023-12-01T06:25:25.183 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:149: TEST_truncated_sna_record: jq -r '.up[0]' 2023-12-01T06:25:25.516 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:149: TEST_truncated_sna_record: local cur_prim=1 2023-12-01T06:25:25.516 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:150: TEST_truncated_sna_record: ceph pg dump pgs 2023-12-01T06:25:25.823 INFO:tasks.workunit.client.0.smithi170.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2023-12-01T06:25:25.823 INFO:tasks.workunit.client.0.smithi170.stdout:1.3 4 0 0 0 0 24 0 0 7 0 7 peering 2023-12-01T06:25:21.729403+0000 26'7 36:46 [1,2,0] 1 [1,2,0] 1 26'7 2023-12-01T06:24:37.079007+0000 26'7 2023-12-01T06:24:37.079007+0000 0 1 no scrub is scheduled 4 0 2023-12-01T06:25:25.824 INFO:tasks.workunit.client.0.smithi170.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:25:22.598583+0000 0'0 39:36 [0,1,2] 0 [0,1,2] 0 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:25:25.824 INFO:tasks.workunit.client.0.smithi170.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2023-12-01T06:25:17.598450+0000 0'0 35:16 [0,1] 0 [0,1] 0 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:25:25.824 INFO:tasks.workunit.client.0.smithi170.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 peering 2023-12-01T06:25:21.729300+0000 0'0 36:20 [1,0,2] 1 [1,0,2] 1 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 no scrub is scheduled 0 0 2023-12-01T06:25:25.824 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:25.824 INFO:tasks.workunit.client.0.smithi170.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2023-12-01T06:25:25.824 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:25:25.840 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:151: TEST_truncated_sna_record: sleep 2 2023-12-01T06:25:27.842 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:152: TEST_truncated_sna_record: ceph pg 1.3 deep-scrub 2023-12-01T06:25:27.979 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:25:27.979 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": true, 2023-12-01T06:25:27.979 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:25:27.979 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:25:27.979 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:25:27.993 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:153: TEST_truncated_sna_record: sleep 5 2023-12-01T06:25:32.995 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:154: TEST_truncated_sna_record: ceph pg dump pgs 2023-12-01T06:25:33.273 INFO:tasks.workunit.client.0.smithi170.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2023-12-01T06:25:33.273 INFO:tasks.workunit.client.0.smithi170.stdout:1.3 4 0 0 0 0 24 0 0 7 0 7 active+clean 2023-12-01T06:25:28.647996+0000 26'7 41:68 [1,2,0] 1 [1,2,0] 1 26'7 2023-12-01T06:25:28.647694+0000 26'7 2023-12-01T06:25:28.647694+0000 0 1 periodic scrub scheduled @ 2023-12-02T06:25:28.647694+0000 4 0 2023-12-01T06:25:33.273 INFO:tasks.workunit.client.0.smithi170.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:25:22.598583+0000 0'0 40:38 [0,1,2] 0 [0,1,2] 0 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:25:33.273 INFO:tasks.workunit.client.0.smithi170.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:25:22.136494+0000 0'0 41:26 [2,0,1] 2 [2,0,1] 2 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:25:33.273 INFO:tasks.workunit.client.0.smithi170.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:25:22.682247+0000 0'0 41:32 [1,0,2] 1 [1,0,2] 1 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:25:33.273 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:33.273 INFO:tasks.workunit.client.0.smithi170.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2023-12-01T06:25:33.274 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:25:33.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:155: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2023-12-01T06:25:33.289 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:155: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2023-12-01T06:25:33.290 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:25:28.642+0000 7fa0d84b2640 -1 log_channel(cluster) log [ERR] : osd.1 found snap mapper error on pg 1.3 oid 1:f5ae0ba1:::objxxx:3 snaps in mapper: {}, oi: {3} ...repaired 2023-12-01T06:25:33.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:156: TEST_truncated_sna_record: grep -a -q ERR td/osd-mapper/osd.1.log 2023-12-01T06:25:33.293 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:160: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2023-12-01T06:25:33.293 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:160: TEST_truncated_sna_record: wc -l 2023-12-01T06:25:33.295 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:160: TEST_truncated_sna_record: local prev_err_cnt=1 2023-12-01T06:25:33.296 INFO:tasks.workunit.client.0.smithi170.stdout:prev count: 1 2023-12-01T06:25:33.296 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:161: TEST_truncated_sna_record: echo 'prev count: 1' 2023-12-01T06:25:33.296 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:164: TEST_truncated_sna_record: ceph pg 1.3 deep-scrub 2023-12-01T06:25:33.427 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:25:33.427 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": true, 2023-12-01T06:25:33.427 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:25:33.427 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:25:33.427 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:25:33.440 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:165: TEST_truncated_sna_record: sleep 5 2023-12-01T06:25:38.442 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:166: TEST_truncated_sna_record: ceph pg dump pgs 2023-12-01T06:25:38.734 INFO:tasks.workunit.client.0.smithi170.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2023-12-01T06:25:38.734 INFO:tasks.workunit.client.0.smithi170.stdout:1.3 4 0 0 0 0 24 0 0 7 0 7 active+clean 2023-12-01T06:25:33.607384+0000 26'7 41:77 [1,2,0] 1 [1,2,0] 1 26'7 2023-12-01T06:25:33.607184+0000 26'7 2023-12-01T06:25:33.607184+0000 0 1 periodic scrub scheduled @ 2023-12-02T06:25:33.607184+0000 4 0 2023-12-01T06:25:38.735 INFO:tasks.workunit.client.0.smithi170.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:25:22.598583+0000 0'0 40:38 [0,1,2] 0 [0,1,2] 0 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:25:38.735 INFO:tasks.workunit.client.0.smithi170.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:25:22.136494+0000 0'0 41:26 [2,0,1] 2 [2,0,1] 2 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:25:38.735 INFO:tasks.workunit.client.0.smithi170.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:25:22.682247+0000 0'0 41:32 [1,0,2] 1 [1,0,2] 1 0'0 2023-12-01T06:24:26.280127+0000 0'0 2023-12-01T06:24:26.280127+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:24:26.280127+0000 0 0 2023-12-01T06:25:38.735 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:25:38.735 INFO:tasks.workunit.client.0.smithi170.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2023-12-01T06:25:38.735 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:25:38.751 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:167: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2023-12-01T06:25:38.751 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:167: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2023-12-01T06:25:38.754 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:25:28.642+0000 7fa0d84b2640 -1 log_channel(cluster) log [ERR] : osd.1 found snap mapper error on pg 1.3 oid 1:f5ae0ba1:::objxxx:3 snaps in mapper: {}, oi: {3} ...repaired 2023-12-01T06:25:38.755 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:168: TEST_truncated_sna_record: wc -l 2023-12-01T06:25:38.755 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:168: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2023-12-01T06:25:38.757 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:168: TEST_truncated_sna_record: local current_err_cnt=1 2023-12-01T06:25:38.757 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:169: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2023-12-01T06:25:38.757 INFO:tasks.workunit.client.0.smithi170.stdout:current count: 1 2023-12-01T06:25:38.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:169: TEST_truncated_sna_record: echo 'current count: 1' 2023-12-01T06:25:38.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:170: TEST_truncated_sna_record: (( current_err_cnt == prev_err_cnt )) 2023-12-01T06:25:38.759 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:171: TEST_truncated_sna_record: kill_daemons td/osd-mapper TERM osd 2023-12-01T06:25:38.759 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-12-01T06:25:38.759 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-12-01T06:25:38.759 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-12-01T06:25:38.759 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-12-01T06:25:38.759 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-12-01T06:25:39.067 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-12-01T06:25:39.067 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:172: TEST_truncated_sna_record: kvdir=td/osd-mapper/1 2023-12-01T06:25:39.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:173: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2023-12-01T06:25:39.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:173: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2023-12-01T06:25:39.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:173: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_' 2023-12-01T06:25:39.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:174: TEST_truncated_sna_record: awk -e '{print $2;}' 2023-12-01T06:25:40.382 INFO:tasks.workunit.client.0.smithi170.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:25:40.382 INFO:tasks.workunit.client.0.smithi170.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2023-12-01T06:25:40.383 INFO:tasks.workunit.client.0.smithi170.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2023-12-01T06:25:40.383 INFO:tasks.workunit.client.0.smithi170.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2023-12-01T06:25:40.383 INFO:tasks.workunit.client.0.smithi170.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:25:40.383 INFO:tasks.workunit.client.0.smithi170.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2023-12-01T06:25:40.384 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2023-12-01T06:25:40.384 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_000000000000000[0-9]_000000000000000' 2023-12-01T06:25:40.385 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: awk -e '{print $2;}' 2023-12-01T06:25:40.385 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: wc -l 2023-12-01T06:25:41.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: local num_sna_full=5 2023-12-01T06:25:41.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:177: TEST_truncated_sna_record: (( num_sna_full == num_sna_b4 )) 2023-12-01T06:25:41.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:178: TEST_truncated_sna_record: return 0 2023-12-01T06:25:41.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:24: run: teardown td/osd-mapper 2023-12-01T06:25:41.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-mapper 2023-12-01T06:25:41.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-12-01T06:25:41.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-mapper KILL 2023-12-01T06:25:41.461 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-12-01T06:25:41.461 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-12-01T06:25:41.461 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-12-01T06:25:41.461 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-12-01T06:25:41.461 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-12-01T06:25:41.571 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-12-01T06:25:41.572 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-12-01T06:25:41.573 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-12-01T06:25:41.573 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-12-01T06:25:41.574 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-12-01T06:25:41.574 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-12-01T06:25:41.574 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-12-01T06:25:41.575 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:25:41.575 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-12-01T06:25:41.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-12-01T06:25:41.576 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:25:41.577 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-12-01T06:25:41.578 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-12-01T06:25:41.578 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-mapper 2023-12-01T06:25:41.595 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-12-01T06:25:41.595 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:25:41.596 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.30922 2023-12-01T06:25:41.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.30922 2023-12-01T06:25:41.598 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-12-01T06:25:41.598 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-12-01T06:25:41.598 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2314: main: code=0 2023-12-01T06:25:41.598 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2318: main: teardown td/osd-mapper 0 2023-12-01T06:25:41.598 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-mapper 2023-12-01T06:25:41.598 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2023-12-01T06:25:41.598 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-mapper KILL 2023-12-01T06:25:41.598 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-12-01T06:25:41.598 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-12-01T06:25:41.598 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-12-01T06:25:41.599 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-12-01T06:25:41.599 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-12-01T06:25:41.600 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-12-01T06:25:41.600 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-12-01T06:25:41.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-12-01T06:25:41.601 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-12-01T06:25:41.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-12-01T06:25:41.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-12-01T06:25:41.602 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-12-01T06:25:41.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:25:41.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-12-01T06:25:41.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-12-01T06:25:41.604 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:25:41.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-12-01T06:25:41.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2023-12-01T06:25:41.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-mapper 2023-12-01T06:25:41.607 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-12-01T06:25:41.607 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:25:41.607 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.30922 2023-12-01T06:25:41.607 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.30922 2023-12-01T06:25:41.608 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-12-01T06:25:41.608 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-12-01T06:25:41.608 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2319: main: return 0 2023-12-01T06:25:41.609 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-01T06:25:41.609 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-01T06:25:41.619 INFO:tasks.workunit:Running workunit scrub/osd-recovery-scrub.sh... 2023-12-01T06:25:41.619 DEBUG:teuthology.orchestra.run.smithi170:workunit test scrub/osd-recovery-scrub.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bbeb098e76b9019fafbe1c7bd55e48604c324c89 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh 2023-12-01T06:25:41.672 INFO:tasks.workunit.client.0.smithi170.stderr:stty: 'standard input': Inappropriate ioctl for device 2023-12-01T06:25:41.678 INFO:tasks.workunit.client.0.smithi170.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2023-12-01T06:25:41.678 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2307: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2023-12-01T06:25:41.678 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2307: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2023-12-01T06:25:41.678 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2308: main: export PYTHONWARNINGS=ignore 2023-12-01T06:25:41.678 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2308: main: PYTHONWARNINGS=ignore 2023-12-01T06:25:41.678 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: main: export CEPH_CONF=/dev/null 2023-12-01T06:25:41.678 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: main: CEPH_CONF=/dev/null 2023-12-01T06:25:41.678 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2310: main: unset CEPH_ARGS 2023-12-01T06:25:41.678 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2312: main: local code 2023-12-01T06:25:41.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2313: main: run td/osd-recovery-scrub 2023-12-01T06:25:41.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:20: run: local dir=td/osd-recovery-scrub 2023-12-01T06:25:41.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:21: run: shift 2023-12-01T06:25:41.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:23: run: export CEPH_MON=127.0.0.1:7124 2023-12-01T06:25:41.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:23: run: CEPH_MON=127.0.0.1:7124 2023-12-01T06:25:41.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:24: run: export CEPH_ARGS 2023-12-01T06:25:41.679 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:25: run: uuidgen 2023-12-01T06:25:41.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:25: run: CEPH_ARGS+='--fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none ' 2023-12-01T06:25:41.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:26: run: CEPH_ARGS+='--mon-host=127.0.0.1:7124 ' 2023-12-01T06:25:41.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:27: run: CEPH_ARGS+='--osd-op-queue=wpq ' 2023-12-01T06:25:41.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:29: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2023-12-01T06:25:41.680 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:30: run: set 2023-12-01T06:25:41.680 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2023-12-01T06:25:41.682 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:30: run: local 'funcs=TEST_recovery_scrub_1 2023-12-01T06:25:41.682 INFO:tasks.workunit.client.0.smithi170.stderr:TEST_recovery_scrub_2' 2023-12-01T06:25:41.682 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:31: run: for func in $funcs 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:32: run: TEST_recovery_scrub_1 td/osd-recovery-scrub 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:39: TEST_recovery_scrub_1: local dir=td/osd-recovery-scrub 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:40: TEST_recovery_scrub_1: local poolname=test 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:42: TEST_recovery_scrub_1: TESTDATA=testdata.47198 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:43: TEST_recovery_scrub_1: OSDS=4 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:44: TEST_recovery_scrub_1: PGS=1 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:45: TEST_recovery_scrub_1: OBJECTS=100 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:46: TEST_recovery_scrub_1: ERRORS=0 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:48: TEST_recovery_scrub_1: setup td/osd-recovery-scrub 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-recovery-scrub 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-recovery-scrub 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-recovery-scrub 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-recovery-scrub KILL 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-12-01T06:25:41.683 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-12-01T06:25:41.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-12-01T06:25:41.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-12-01T06:25:41.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-12-01T06:25:41.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-12-01T06:25:41.684 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-12-01T06:25:41.685 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-12-01T06:25:41.685 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-12-01T06:25:41.686 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-12-01T06:25:41.686 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-12-01T06:25:41.686 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-12-01T06:25:41.687 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:25:41.687 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-12-01T06:25:41.688 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-12-01T06:25:41.688 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:25:41.688 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-12-01T06:25:41.690 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-12-01T06:25:41.690 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-recovery-scrub 2023-12-01T06:25:41.691 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-12-01T06:25:41.691 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:25:41.691 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:25:41.691 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.47198 2023-12-01T06:25:41.692 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-12-01T06:25:41.692 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-12-01T06:25:41.692 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-recovery-scrub 2023-12-01T06:25:41.693 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2023-12-01T06:25:41.693 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:25:41.693 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:25:41.694 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.47198 2023-12-01T06:25:41.695 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2023-12-01T06:25:41.695 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2023-12-01T06:25:41.696 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2023-12-01T06:25:41.696 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2023-12-01T06:25:41.696 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-recovery-scrub 1' TERM HUP INT 2023-12-01T06:25:41.696 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:49: TEST_recovery_scrub_1: run_mon td/osd-recovery-scrub a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2023-12-01T06:25:41.696 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-recovery-scrub 2023-12-01T06:25:41.696 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2023-12-01T06:25:41.696 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2023-12-01T06:25:41.696 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2023-12-01T06:25:41.696 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-recovery-scrub/a 2023-12-01T06:25:41.696 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-recovery-scrub/a --run-dir=td/osd-recovery-scrub --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2023-12-01T06:25:42.004 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2023-12-01T06:25:42.004 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:25:42.004 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:25:42.004 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:25:42.004 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:25:42.004 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:25:42.005 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:25:42.005 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-recovery-scrub/a '--log-file=td/osd-recovery-scrub/$name.log' '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --mon-cluster-log-file=td/osd-recovery-scrub/log --run-dir=td/osd-recovery-scrub '--pid-file=td/osd-recovery-scrub/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2023-12-01T06:25:42.216 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2023-12-01T06:25:42.216 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2023-12-01T06:25:42.216 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2023-12-01T06:25:42.216 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2023-12-01T06:25:42.217 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2023-12-01T06:25:42.217 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2023-12-01T06:25:42.218 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2023-12-01T06:25:42.218 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-12-01T06:25:42.218 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-12-01T06:25:42.218 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:25:42.218 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:25:42.218 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:25:42.218 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.47198/ceph-mon.a.asok 2023-12-01T06:25:42.218 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2023-12-01T06:25:42.219 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.47198/ceph-mon.a.asok config get fsid 2023-12-01T06:25:42.335 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2023-12-01T06:25:42.335 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2023-12-01T06:25:42.335 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2023-12-01T06:25:42.335 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2023-12-01T06:25:42.336 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2023-12-01T06:25:42.336 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2023-12-01T06:25:42.336 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-12-01T06:25:42.336 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-12-01T06:25:42.336 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:25:42.336 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:25:42.336 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:25:42.336 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.47198/ceph-mon.a.asok 2023-12-01T06:25:42.337 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2023-12-01T06:25:42.337 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.47198/ceph-mon.a.asok config get mon_host 2023-12-01T06:25:42.439 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:50: TEST_recovery_scrub_1: run_mgr td/osd-recovery-scrub x 2023-12-01T06:25:42.439 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-recovery-scrub 2023-12-01T06:25:42.439 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2023-12-01T06:25:42.439 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2023-12-01T06:25:42.439 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2023-12-01T06:25:42.439 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-recovery-scrub/x 2023-12-01T06:25:42.439 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2023-12-01T06:25:43.304 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2023-12-01T06:25:43.305 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:25:43.305 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:25:43.305 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:25:43.305 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:25:43.305 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:25:43.306 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:25:43.306 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-12-01T06:25:43.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-recovery-scrub/x '--log-file=td/osd-recovery-scrub/$name.log' '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --run-dir=td/osd-recovery-scrub '--pid-file=td/osd-recovery-scrub/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-12-01T06:25:43.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:51: TEST_recovery_scrub_1: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 ' 2023-12-01T06:25:43.341 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:52: TEST_recovery_scrub_1: ceph_osd_args+='--osd_scrub_backoff_ratio=0 ' 2023-12-01T06:25:43.341 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:53: TEST_recovery_scrub_1: ceph_osd_args+='--osd_stats_update_period_not_scrubbing=3 ' 2023-12-01T06:25:43.341 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:54: TEST_recovery_scrub_1: ceph_osd_args+=--osd_stats_update_period_scrubbing=2 2023-12-01T06:25:43.341 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: expr 4 - 1 2023-12-01T06:25:43.341 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: seq 0 3 2023-12-01T06:25:43.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:25:43.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 0 --osd_scrub_during_recovery=false 2023-12-01T06:25:43.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2023-12-01T06:25:43.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:25:43.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2023-12-01T06:25:43.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:25:43.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/0 2023-12-01T06:25:43.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2023-12-01T06:25:43.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:25:43.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:25:43.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:25:43.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/0' 2023-12-01T06:25:43.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/0/journal' 2023-12-01T06:25:43.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:25:43.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:25:43.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2023-12-01T06:25:43.346 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:25:43.346 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:25:43.346 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:25:43.346 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:25:43.346 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:25:43.346 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:25:43.348 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:25:43.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:25:43.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:25:43.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:25:43.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:25:43.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2023-12-01T06:25:43.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2023-12-01T06:25:43.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:25:43.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:25:43.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:25:43.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:25:43.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:25:43.349 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2023-12-01T06:25:43.349 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/0 2023-12-01T06:25:43.349 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:25:43.349 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=89db0373-6200-4c56-8a1a-5196192c6345 2023-12-01T06:25:43.349 INFO:tasks.workunit.client.0.smithi170.stdout:add osd0 89db0373-6200-4c56-8a1a-5196192c6345 2023-12-01T06:25:43.350 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 89db0373-6200-4c56-8a1a-5196192c6345' 2023-12-01T06:25:43.350 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:25:43.364 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBnfGllWj+lFRAAlOssMBrKWhM1i5CQkhH4yA== 2023-12-01T06:25:43.364 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBnfGllWj+lFRAAlOssMBrKWhM1i5CQkhH4yA=="}' 2023-12-01T06:25:43.364 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 89db0373-6200-4c56-8a1a-5196192c6345 -i td/osd-recovery-scrub/0/new.json 2023-12-01T06:25:43.599 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:25:43.614 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/0/new.json 2023-12-01T06:25:43.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/0 --osd-journal=td/osd-recovery-scrub/0/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQBnfGllWj+lFRAAlOssMBrKWhM1i5CQkhH4yA== --osd-uuid 89db0373-6200-4c56-8a1a-5196192c6345 2023-12-01T06:25:43.650 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:43.646+0000 7fa6a621c640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:43.652 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:43.650+0000 7fa6a621c640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:43.653 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:43.650+0000 7fa6a621c640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:43.653 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:43.650+0000 7fa6a621c640 -1 bluestore(td/osd-recovery-scrub/0/block) _read_bdev_label failed to open td/osd-recovery-scrub/0/block: (2) No such file or directory 2023-12-01T06:25:43.654 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:43.650+0000 7fa6a621c640 -1 bluestore(td/osd-recovery-scrub/0/block) _read_bdev_label failed to open td/osd-recovery-scrub/0/block: (2) No such file or directory 2023-12-01T06:25:43.654 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:43.650+0000 7fa6a621c640 -1 bluestore(td/osd-recovery-scrub/0/block) _read_bdev_label failed to open td/osd-recovery-scrub/0/block: (2) No such file or directory 2023-12-01T06:25:43.733 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:43.730+0000 7fa6a621c640 -1 bluestore(td/osd-recovery-scrub/0) _read_fsid unparsable uuid 2023-12-01T06:25:49.837 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/0/keyring 2023-12-01T06:25:49.837 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:25:49.838 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd0 key to auth repository 2023-12-01T06:25:49.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2023-12-01T06:25:49.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:25:50.213 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2023-12-01T06:25:50.213 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.0 2023-12-01T06:25:50.214 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/0 --osd-journal=td/osd-recovery-scrub/0/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2023-12-01T06:25:50.214 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:25:50.214 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:25:50.214 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:25:50.247 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:50.242+0000 7fbdd3c00640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:50.277 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:50.274+0000 7fbdd3c00640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:50.278 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:50.274+0000 7fbdd3c00640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:50.535 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2023-12-01T06:25:50.535 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:25:50.535 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2023-12-01T06:25:50.535 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:25:50.535 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:25:50.535 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:50.535 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:25:50.535 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:25:50.536 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:50.536 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:25:50.839 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:50.833+0000 7fbdd3c00640 -1 Falling back to public interface 2023-12-01T06:25:50.869 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:25:51.870 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:25:51.870 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:51.870 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:25:51.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:25:51.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:51.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:25:52.176 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:25:53.178 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:25:53.178 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:53.178 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:25:53.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:25:53.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:53.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:25:53.358 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:53.353+0000 7fbdd3c00640 -1 osd.0 0 log_to_monitors true 2023-12-01T06:25:53.532 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:25:54.533 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:25:54.534 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:25:54.534 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:25:54.534 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:25:54.534 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:25:54.535 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:25:54.874 INFO:tasks.workunit.client.0.smithi170.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/232366513,v1:127.0.0.1:6803/232366513] [v2:127.0.0.1:6804/232366513,v1:127.0.0.1:6805/232366513] exists,up 89db0373-6200-4c56-8a1a-5196192c6345 2023-12-01T06:25:54.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:25:54.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:25:54.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:25:54.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:25:54.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 1 --osd_scrub_during_recovery=false 2023-12-01T06:25:54.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2023-12-01T06:25:54.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:25:54.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2023-12-01T06:25:54.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:25:54.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/1 2023-12-01T06:25:54.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2023-12-01T06:25:54.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:25:54.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:25:54.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:25:54.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/1' 2023-12-01T06:25:54.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/1/journal' 2023-12-01T06:25:54.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:25:54.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:25:54.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2023-12-01T06:25:54.879 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:25:54.879 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:25:54.879 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:25:54.879 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:25:54.879 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:25:54.879 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:25:54.880 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:25:54.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:25:54.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:25:54.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:25:54.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:25:54.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2023-12-01T06:25:54.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2023-12-01T06:25:54.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:25:54.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:25:54.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:25:54.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:25:54.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:25:54.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2023-12-01T06:25:54.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/1 2023-12-01T06:25:54.881 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:25:54.881 INFO:tasks.workunit.client.0.smithi170.stdout:add osd1 854dc59f-1ce6-4714-8a57-dda29f0b13df 2023-12-01T06:25:54.882 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=854dc59f-1ce6-4714-8a57-dda29f0b13df 2023-12-01T06:25:54.882 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 854dc59f-1ce6-4714-8a57-dda29f0b13df' 2023-12-01T06:25:54.882 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:25:54.900 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQByfGllpdSkNRAAMrt8uEDZVT3RTbS+fpHKGA== 2023-12-01T06:25:54.900 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQByfGllpdSkNRAAMrt8uEDZVT3RTbS+fpHKGA=="}' 2023-12-01T06:25:54.900 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 854dc59f-1ce6-4714-8a57-dda29f0b13df -i td/osd-recovery-scrub/1/new.json 2023-12-01T06:25:55.240 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:25:55.257 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/1/new.json 2023-12-01T06:25:55.259 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/1 --osd-journal=td/osd-recovery-scrub/1/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQByfGllpdSkNRAAMrt8uEDZVT3RTbS+fpHKGA== --osd-uuid 854dc59f-1ce6-4714-8a57-dda29f0b13df 2023-12-01T06:25:55.290 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:55.285+0000 7f6b8217d640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:55.325 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:55.321+0000 7f6b8217d640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:55.326 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:55.321+0000 7f6b8217d640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:25:55.326 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:55.321+0000 7f6b8217d640 -1 bluestore(td/osd-recovery-scrub/1/block) _read_bdev_label failed to open td/osd-recovery-scrub/1/block: (2) No such file or directory 2023-12-01T06:25:55.326 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:55.321+0000 7f6b8217d640 -1 bluestore(td/osd-recovery-scrub/1/block) _read_bdev_label failed to open td/osd-recovery-scrub/1/block: (2) No such file or directory 2023-12-01T06:25:55.326 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:55.321+0000 7f6b8217d640 -1 bluestore(td/osd-recovery-scrub/1/block) _read_bdev_label failed to open td/osd-recovery-scrub/1/block: (2) No such file or directory 2023-12-01T06:25:55.391 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:25:55.385+0000 7f6b8217d640 -1 bluestore(td/osd-recovery-scrub/1) _read_fsid unparsable uuid 2023-12-01T06:26:01.329 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/1/keyring 2023-12-01T06:26:01.329 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:26:01.331 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd1 key to auth repository 2023-12-01T06:26:01.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2023-12-01T06:26:01.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:26:01.711 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2023-12-01T06:26:01.711 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.1 2023-12-01T06:26:01.712 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/1 --osd-journal=td/osd-recovery-scrub/1/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2023-12-01T06:26:01.712 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:26:01.712 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:26:01.712 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:26:01.744 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:01.741+0000 7f2618fe5640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:01.771 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:01.765+0000 7f2618fe5640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:01.773 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:01.769+0000 7f2618fe5640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:02.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2023-12-01T06:26:02.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:26:02.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2023-12-01T06:26:02.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:26:02.041 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:26:02.042 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:26:02.042 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:26:02.042 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:26:02.042 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:26:02.042 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:26:02.325 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:02.317+0000 7f2618fe5640 -1 Falling back to public interface 2023-12-01T06:26:02.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:26:03.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:26:03.346 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:26:03.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:26:03.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:26:03.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:26:03.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:26:03.642 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:26:04.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:26:04.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:26:04.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:26:04.644 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:26:04.645 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:26:04.645 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:26:04.890 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:04.885+0000 7f2618fe5640 -1 osd.1 0 log_to_monitors true 2023-12-01T06:26:04.960 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:26:05.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:26:05.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:26:05.962 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:26:05.962 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:26:05.962 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:26:05.962 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:26:06.309 INFO:tasks.workunit.client.0.smithi170.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3775625476,v1:127.0.0.1:6811/3775625476] [v2:127.0.0.1:6812/3775625476,v1:127.0.0.1:6813/3775625476] exists,up 854dc59f-1ce6-4714-8a57-dda29f0b13df 2023-12-01T06:26:06.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:26:06.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:26:06.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:26:06.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:26:06.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 2 --osd_scrub_during_recovery=false 2023-12-01T06:26:06.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2023-12-01T06:26:06.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:26:06.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2023-12-01T06:26:06.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:26:06.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/2 2023-12-01T06:26:06.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2023-12-01T06:26:06.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:26:06.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:26:06.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:26:06.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/2' 2023-12-01T06:26:06.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/2/journal' 2023-12-01T06:26:06.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:26:06.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:26:06.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2023-12-01T06:26:06.312 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:26:06.312 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:26:06.312 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:26:06.314 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:26:06.314 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:26:06.314 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:26:06.314 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:26:06.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:26:06.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:26:06.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:26:06.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:26:06.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2023-12-01T06:26:06.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2023-12-01T06:26:06.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:26:06.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:26:06.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:26:06.315 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:26:06.315 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:26:06.315 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2023-12-01T06:26:06.315 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/2 2023-12-01T06:26:06.315 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:26:06.315 INFO:tasks.workunit.client.0.smithi170.stdout:add osd2 e3422aa7-b7ab-45e8-9e5c-20d1d1842b58 2023-12-01T06:26:06.315 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e3422aa7-b7ab-45e8-9e5c-20d1d1842b58 2023-12-01T06:26:06.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 e3422aa7-b7ab-45e8-9e5c-20d1d1842b58' 2023-12-01T06:26:06.316 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:26:06.336 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB+fGllNSULFBAAQtkPsx5GeOoJkpFI91TxOA== 2023-12-01T06:26:06.337 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB+fGllNSULFBAAQtkPsx5GeOoJkpFI91TxOA=="}' 2023-12-01T06:26:06.337 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e3422aa7-b7ab-45e8-9e5c-20d1d1842b58 -i td/osd-recovery-scrub/2/new.json 2023-12-01T06:26:06.686 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:26:06.703 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/2/new.json 2023-12-01T06:26:06.704 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/2 --osd-journal=td/osd-recovery-scrub/2/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQB+fGllNSULFBAAQtkPsx5GeOoJkpFI91TxOA== --osd-uuid e3422aa7-b7ab-45e8-9e5c-20d1d1842b58 2023-12-01T06:26:06.733 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:06.729+0000 7fb05cc90640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:06.763 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:06.757+0000 7fb05cc90640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:06.764 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:06.761+0000 7fb05cc90640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:06.764 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:06.761+0000 7fb05cc90640 -1 bluestore(td/osd-recovery-scrub/2/block) _read_bdev_label failed to open td/osd-recovery-scrub/2/block: (2) No such file or directory 2023-12-01T06:26:06.764 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:06.761+0000 7fb05cc90640 -1 bluestore(td/osd-recovery-scrub/2/block) _read_bdev_label failed to open td/osd-recovery-scrub/2/block: (2) No such file or directory 2023-12-01T06:26:06.764 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:06.761+0000 7fb05cc90640 -1 bluestore(td/osd-recovery-scrub/2/block) _read_bdev_label failed to open td/osd-recovery-scrub/2/block: (2) No such file or directory 2023-12-01T06:26:06.828 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:06.821+0000 7fb05cc90640 -1 bluestore(td/osd-recovery-scrub/2) _read_fsid unparsable uuid 2023-12-01T06:26:12.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/2/keyring 2023-12-01T06:26:12.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:26:12.681 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd2 key to auth repository 2023-12-01T06:26:12.681 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2023-12-01T06:26:12.681 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:26:13.065 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.2 2023-12-01T06:26:13.066 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2023-12-01T06:26:13.066 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/2 --osd-journal=td/osd-recovery-scrub/2/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2023-12-01T06:26:13.066 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:26:13.066 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:26:13.067 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:26:13.094 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:13.089+0000 7f62c2a66640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:13.096 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:13.093+0000 7f62c2a66640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:13.097 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:13.093+0000 7f62c2a66640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:13.383 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2023-12-01T06:26:13.383 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:26:13.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2023-12-01T06:26:13.384 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:26:13.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:26:13.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:26:13.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:26:13.385 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:26:13.385 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:26:13.385 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:26:13.649 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:13.645+0000 7f62c2a66640 -1 Falling back to public interface 2023-12-01T06:26:13.706 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:26:14.707 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:26:14.708 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:26:14.708 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:26:14.708 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:26:14.708 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:26:14.708 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:26:15.012 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:26:16.013 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:26:16.014 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:26:16.014 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:26:16.014 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:26:16.014 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:26:16.015 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:26:16.209 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:16.205+0000 7f62c2a66640 -1 osd.2 0 log_to_monitors true 2023-12-01T06:26:16.319 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:26:17.321 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:26:17.321 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:26:17.321 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:26:17.322 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:26:17.322 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:26:17.322 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:26:17.660 INFO:tasks.workunit.client.0.smithi170.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2052225580,v1:127.0.0.1:6819/2052225580] [v2:127.0.0.1:6820/2052225580,v1:127.0.0.1:6821/2052225580] exists,up e3422aa7-b7ab-45e8-9e5c-20d1d1842b58 2023-12-01T06:26:17.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:26:17.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:26:17.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:26:17.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:26:17.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 3 --osd_scrub_during_recovery=false 2023-12-01T06:26:17.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2023-12-01T06:26:17.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:26:17.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2023-12-01T06:26:17.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:26:17.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/3 2023-12-01T06:26:17.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2023-12-01T06:26:17.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:26:17.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:26:17.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:26:17.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/3' 2023-12-01T06:26:17.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/3/journal' 2023-12-01T06:26:17.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:26:17.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:26:17.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2023-12-01T06:26:17.663 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:26:17.663 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:26:17.663 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:26:17.664 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:26:17.664 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/3 2023-12-01T06:26:17.665 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:26:17.666 INFO:tasks.workunit.client.0.smithi170.stdout:add osd3 69800b91-d641-48da-b215-2e52d5a5b966 2023-12-01T06:26:17.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=69800b91-d641-48da-b215-2e52d5a5b966 2023-12-01T06:26:17.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 69800b91-d641-48da-b215-2e52d5a5b966' 2023-12-01T06:26:17.666 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:26:17.688 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCJfGlls90AKRAAxr8ql/meXwqpo4uRgkfeUA== 2023-12-01T06:26:17.688 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCJfGlls90AKRAAxr8ql/meXwqpo4uRgkfeUA=="}' 2023-12-01T06:26:17.688 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 69800b91-d641-48da-b215-2e52d5a5b966 -i td/osd-recovery-scrub/3/new.json 2023-12-01T06:26:18.068 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:26:18.087 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/3/new.json 2023-12-01T06:26:18.088 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/3 --osd-journal=td/osd-recovery-scrub/3/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQCJfGlls90AKRAAxr8ql/meXwqpo4uRgkfeUA== --osd-uuid 69800b91-d641-48da-b215-2e52d5a5b966 2023-12-01T06:26:18.129 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:18.121+0000 7ff4ca1b2640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:18.144 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:18.141+0000 7ff4ca1b2640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:18.146 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:18.141+0000 7ff4ca1b2640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:18.146 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:18.141+0000 7ff4ca1b2640 -1 bluestore(td/osd-recovery-scrub/3/block) _read_bdev_label failed to open td/osd-recovery-scrub/3/block: (2) No such file or directory 2023-12-01T06:26:18.146 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:18.141+0000 7ff4ca1b2640 -1 bluestore(td/osd-recovery-scrub/3/block) _read_bdev_label failed to open td/osd-recovery-scrub/3/block: (2) No such file or directory 2023-12-01T06:26:18.146 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:18.141+0000 7ff4ca1b2640 -1 bluestore(td/osd-recovery-scrub/3/block) _read_bdev_label failed to open td/osd-recovery-scrub/3/block: (2) No such file or directory 2023-12-01T06:26:18.212 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:18.209+0000 7ff4ca1b2640 -1 bluestore(td/osd-recovery-scrub/3) _read_fsid unparsable uuid 2023-12-01T06:26:23.982 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/3/keyring 2023-12-01T06:26:23.983 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:26:23.984 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd3 key to auth repository 2023-12-01T06:26:23.984 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2023-12-01T06:26:23.984 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:26:24.371 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.3 2023-12-01T06:26:24.371 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2023-12-01T06:26:24.371 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/3 --osd-journal=td/osd-recovery-scrub/3/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2023-12-01T06:26:24.371 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:26:24.372 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:26:24.372 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:26:24.398 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:24.393+0000 7f3deb5c5640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:24.418 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:24.413+0000 7f3deb5c5640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:24.419 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:24.417+0000 7f3deb5c5640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:26:24.681 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2023-12-01T06:26:24.682 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:26:24.682 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2023-12-01T06:26:24.682 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:26:24.682 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:26:24.682 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:26:24.682 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:26:24.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:26:24.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:26:24.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2023-12-01T06:26:24.973 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:24.969+0000 7f3deb5c5640 -1 Falling back to public interface 2023-12-01T06:26:24.974 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:26:25.975 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:26:25.976 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:26:25.976 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:26:25.976 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:26:25.977 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:26:25.977 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2023-12-01T06:26:26.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:26:27.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:26:27.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:26:27.278 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:26:27.279 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:26:27.279 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:26:27.279 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2023-12-01T06:26:27.497 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:26:27.493+0000 7f3deb5c5640 -1 osd.3 0 log_to_monitors true 2023-12-01T06:26:27.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:26:28.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:26:28.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:26:28.595 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:26:28.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:26:28.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:26:28.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2023-12-01T06:26:28.957 INFO:tasks.workunit.client.0.smithi170.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3731449950,v1:127.0.0.1:6827/3731449950] [v2:127.0.0.1:6828/3731449950,v1:127.0.0.1:6829/3731449950] exists,up 69800b91-d641-48da-b215-2e52d5a5b966 2023-12-01T06:26:28.958 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:26:28.958 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:26:28.958 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:26:28.958 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:61: TEST_recovery_scrub_1: create_pool test 1 1 2023-12-01T06:26:28.959 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2023-12-01T06:26:29.374 INFO:tasks.workunit.client.0.smithi170.stderr:pool 'test' created 2023-12-01T06:26:29.394 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2023-12-01T06:26:30.396 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:62: TEST_recovery_scrub_1: wait_for_clean 2023-12-01T06:26:30.396 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1642: wait_for_clean: local cmd= 2023-12-01T06:26:30.396 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1643: wait_for_clean: local num_active_clean=-1 2023-12-01T06:26:30.397 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1644: wait_for_clean: local cur_active_clean 2023-12-01T06:26:30.397 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1645: wait_for_clean: get_timeout_delays 90 .1 2023-12-01T06:26:30.398 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1588: get_timeout_delays: shopt -q -o xtrace 2023-12-01T06:26:30.398 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1588: get_timeout_delays: echo true 2023-12-01T06:26:30.399 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1588: get_timeout_delays: local trace=true 2023-12-01T06:26:30.399 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1589: get_timeout_delays: true 2023-12-01T06:26:30.399 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1589: get_timeout_delays: shopt -u -o xtrace 2023-12-01T06:26:30.515 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1645: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2023-12-01T06:26:30.515 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1645: wait_for_clean: local -a delays 2023-12-01T06:26:30.515 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1646: wait_for_clean: local -i loop=0 2023-12-01T06:26:30.515 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: flush_pg_stats 2023-12-01T06:26:30.515 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2197: flush_pg_stats: local timeout=300 2023-12-01T06:26:30.515 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2199: flush_pg_stats: ceph osd ls 2023-12-01T06:26:30.837 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2199: flush_pg_stats: ids='0 2023-12-01T06:26:30.837 INFO:tasks.workunit.client.0.smithi170.stderr:1 2023-12-01T06:26:30.837 INFO:tasks.workunit.client.0.smithi170.stderr:2 2023-12-01T06:26:30.837 INFO:tasks.workunit.client.0.smithi170.stderr:3' 2023-12-01T06:26:30.837 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2200: flush_pg_stats: seqs= 2023-12-01T06:26:30.837 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:26:30.837 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2023-12-01T06:26:30.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=21474836489 2023-12-01T06:26:30.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 21474836489 2023-12-01T06:26:30.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836489' 2023-12-01T06:26:30.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:26:30.987 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2023-12-01T06:26:31.131 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=42949672966 2023-12-01T06:26:31.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 42949672966 2023-12-01T06:26:31.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836489 1-42949672966' 2023-12-01T06:26:31.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:26:31.132 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2023-12-01T06:26:31.282 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=64424509444 2023-12-01T06:26:31.282 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 64424509444 2023-12-01T06:26:31.282 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836489 1-42949672966 2-64424509444' 2023-12-01T06:26:31.282 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:26:31.283 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2023-12-01T06:26:31.425 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=85899345922 2023-12-01T06:26:31.425 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 85899345922 2023-12-01T06:26:31.425 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836489 1-42949672966 2-64424509444 3-85899345922' 2023-12-01T06:26:31.425 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:26:31.426 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 0-21474836489 2023-12-01T06:26:31.426 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:26:31.427 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=0 2023-12-01T06:26:31.427 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 0-21474836489 2023-12-01T06:26:31.427 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:26:31.428 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=21474836489 2023-12-01T06:26:31.428 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2023-12-01T06:26:31.429 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.0 seq 21474836489 2023-12-01T06:26:31.429 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 0 2023-12-01T06:26:31.725 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 21474836488 -lt 21474836489 2023-12-01T06:26:31.725 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2215: flush_pg_stats: sleep 1 2023-12-01T06:26:32.726 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2216: flush_pg_stats: '[' 300 -eq 0 ']' 2023-12-01T06:26:32.728 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 0 2023-12-01T06:26:33.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 21474836489 -lt 21474836489 2023-12-01T06:26:33.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:26:33.042 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 1-42949672966 2023-12-01T06:26:33.042 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:26:33.044 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=1 2023-12-01T06:26:33.045 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 1-42949672966 2023-12-01T06:26:33.045 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:26:33.046 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=42949672966 2023-12-01T06:26:33.046 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2023-12-01T06:26:33.046 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.1 seq 42949672966 2023-12-01T06:26:33.047 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 1 2023-12-01T06:26:33.369 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 42949672967 -lt 42949672966 2023-12-01T06:26:33.369 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:26:33.370 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 2-64424509444 2023-12-01T06:26:33.371 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:26:33.372 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=2 2023-12-01T06:26:33.373 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 2-64424509444 2023-12-01T06:26:33.373 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:26:33.374 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=64424509444 2023-12-01T06:26:33.374 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.2 seq 64424509444 2023-12-01T06:26:33.374 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2023-12-01T06:26:33.374 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 2 2023-12-01T06:26:33.694 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 64424509444 -lt 64424509444 2023-12-01T06:26:33.694 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:26:33.694 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 3-85899345922 2023-12-01T06:26:33.694 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:26:33.694 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=3 2023-12-01T06:26:33.694 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:26:33.694 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 3-85899345922 2023-12-01T06:26:33.695 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=85899345922 2023-12-01T06:26:33.695 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.3 seq 85899345922' 2023-12-01T06:26:33.695 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.3 seq 85899345922 2023-12-01T06:26:33.695 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 3 2023-12-01T06:26:34.024 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 85899345922 -lt 85899345922 2023-12-01T06:26:34.025 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: get_num_pgs 2023-12-01T06:26:34.025 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2023-12-01T06:26:34.026 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2023-12-01T06:26:34.390 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: test 1 == 0 2023-12-01T06:26:34.391 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: true 2023-12-01T06:26:34.391 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: get_num_active_clean 2023-12-01T06:26:34.391 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2023-12-01T06:26:34.391 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2023-12-01T06:26:34.391 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2023-12-01T06:26:34.391 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2023-12-01T06:26:34.391 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2023-12-01T06:26:34.695 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: cur_active_clean=1 2023-12-01T06:26:34.696 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: get_num_pgs 2023-12-01T06:26:34.696 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2023-12-01T06:26:34.696 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2023-12-01T06:26:35.061 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: test 1 = 1 2023-12-01T06:26:35.061 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: break 2023-12-01T06:26:35.061 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: return 0 2023-12-01T06:26:35.062 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: ceph osd dump 2023-12-01T06:26:35.062 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: grep '^pool.*['\'']test['\'']' 2023-12-01T06:26:35.062 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: awk '{ print $2 }' 2023-12-01T06:26:35.372 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: poolid=1 2023-12-01T06:26:35.372 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:65: TEST_recovery_scrub_1: ceph pg dump pgs 2023-12-01T06:26:35.651 INFO:tasks.workunit.client.0.smithi170.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2023-12-01T06:26:35.652 INFO:tasks.workunit.client.0.smithi170.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:26:30.281373+0000 0'0 23:8 [1] 1 [1] 1 0'0 2023-12-01T06:26:29.321857+0000 0'0 2023-12-01T06:26:29.321857+0000 0 0 periodic scrub scheduled @ 2023-12-02T13:20:51.751769+0000 0 0 2023-12-01T06:26:35.652 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:26:35.652 INFO:tasks.workunit.client.0.smithi170.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2023-12-01T06:26:35.652 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:26:35.667 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:67: TEST_recovery_scrub_1: dd if=/dev/urandom of=testdata.47198 bs=1M count=50 2023-12-01T06:26:35.839 INFO:tasks.workunit.client.0.smithi170.stderr:50+0 records in 2023-12-01T06:26:35.839 INFO:tasks.workunit.client.0.smithi170.stderr:50+0 records out 2023-12-01T06:26:35.839 INFO:tasks.workunit.client.0.smithi170.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.170162 s, 308 MB/s 2023-12-01T06:26:35.840 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: seq 1 100 2023-12-01T06:26:35.842 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:35.842 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj1 testdata.47198 2023-12-01T06:26:37.361 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:37.361 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj2 testdata.47198 2023-12-01T06:26:38.876 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:38.876 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj3 testdata.47198 2023-12-01T06:26:40.335 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:40.335 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj4 testdata.47198 2023-12-01T06:26:41.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:41.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj5 testdata.47198 2023-12-01T06:26:43.380 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:43.380 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj6 testdata.47198 2023-12-01T06:26:44.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:44.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj7 testdata.47198 2023-12-01T06:26:46.361 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:46.361 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj8 testdata.47198 2023-12-01T06:26:47.876 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:47.877 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj9 testdata.47198 2023-12-01T06:26:49.324 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:49.324 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj10 testdata.47198 2023-12-01T06:26:50.856 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:50.856 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj11 testdata.47198 2023-12-01T06:26:52.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:52.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj12 testdata.47198 2023-12-01T06:26:53.844 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:53.844 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj13 testdata.47198 2023-12-01T06:26:55.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:55.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj14 testdata.47198 2023-12-01T06:26:56.866 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:56.866 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj15 testdata.47198 2023-12-01T06:26:58.263 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:58.263 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj16 testdata.47198 2023-12-01T06:26:59.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:26:59.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj17 testdata.47198 2023-12-01T06:27:01.292 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:01.292 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj18 testdata.47198 2023-12-01T06:27:02.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:02.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj19 testdata.47198 2023-12-01T06:27:04.282 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:04.282 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj20 testdata.47198 2023-12-01T06:27:06.471 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:06.471 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj21 testdata.47198 2023-12-01T06:27:08.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:08.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj22 testdata.47198 2023-12-01T06:27:09.569 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:09.569 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj23 testdata.47198 2023-12-01T06:27:11.099 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:11.099 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj24 testdata.47198 2023-12-01T06:27:12.608 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:12.608 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj25 testdata.47198 2023-12-01T06:27:14.072 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:14.072 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj26 testdata.47198 2023-12-01T06:27:15.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:15.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj27 testdata.47198 2023-12-01T06:27:17.227 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:17.227 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj28 testdata.47198 2023-12-01T06:27:18.866 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:18.866 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj29 testdata.47198 2023-12-01T06:27:20.357 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:20.357 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj30 testdata.47198 2023-12-01T06:27:21.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:21.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj31 testdata.47198 2023-12-01T06:27:23.469 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:23.469 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj32 testdata.47198 2023-12-01T06:27:25.043 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:25.043 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj33 testdata.47198 2023-12-01T06:27:26.526 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:26.526 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj34 testdata.47198 2023-12-01T06:27:27.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:27.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj35 testdata.47198 2023-12-01T06:27:29.515 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:29.515 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj36 testdata.47198 2023-12-01T06:27:31.028 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:31.028 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj37 testdata.47198 2023-12-01T06:27:32.568 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:32.569 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj38 testdata.47198 2023-12-01T06:27:34.034 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:34.034 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj39 testdata.47198 2023-12-01T06:27:35.531 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:35.531 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj40 testdata.47198 2023-12-01T06:27:37.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:37.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj41 testdata.47198 2023-12-01T06:27:38.819 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:38.819 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj42 testdata.47198 2023-12-01T06:27:40.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:40.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj43 testdata.47198 2023-12-01T06:27:41.801 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:41.801 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj44 testdata.47198 2023-12-01T06:27:43.375 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:43.375 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj45 testdata.47198 2023-12-01T06:27:44.915 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:44.916 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj46 testdata.47198 2023-12-01T06:27:46.379 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:46.379 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj47 testdata.47198 2023-12-01T06:27:47.907 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:47.908 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj48 testdata.47198 2023-12-01T06:27:49.442 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:49.442 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj49 testdata.47198 2023-12-01T06:27:50.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:50.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj50 testdata.47198 2023-12-01T06:27:52.513 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:52.514 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj51 testdata.47198 2023-12-01T06:27:53.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:53.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj52 testdata.47198 2023-12-01T06:27:55.502 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:55.502 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj53 testdata.47198 2023-12-01T06:27:57.031 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:57.032 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj54 testdata.47198 2023-12-01T06:27:58.640 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:27:58.641 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj55 testdata.47198 2023-12-01T06:28:00.103 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:00.104 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj56 testdata.47198 2023-12-01T06:28:01.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:01.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj57 testdata.47198 2023-12-01T06:28:03.175 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:03.175 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj58 testdata.47198 2023-12-01T06:28:04.733 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:04.733 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj59 testdata.47198 2023-12-01T06:28:06.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:06.229 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj60 testdata.47198 2023-12-01T06:28:07.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:07.763 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj61 testdata.47198 2023-12-01T06:28:09.544 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:09.544 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj62 testdata.47198 2023-12-01T06:28:11.059 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:11.060 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj63 testdata.47198 2023-12-01T06:28:12.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:12.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj64 testdata.47198 2023-12-01T06:28:14.155 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:14.155 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj65 testdata.47198 2023-12-01T06:28:15.660 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:15.660 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj66 testdata.47198 2023-12-01T06:28:17.210 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:17.210 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj67 testdata.47198 2023-12-01T06:28:18.791 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:18.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj68 testdata.47198 2023-12-01T06:28:20.231 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:20.231 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj69 testdata.47198 2023-12-01T06:28:21.737 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:21.738 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj70 testdata.47198 2023-12-01T06:28:23.327 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:23.327 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj71 testdata.47198 2023-12-01T06:28:24.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:24.861 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj72 testdata.47198 2023-12-01T06:28:26.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:26.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj73 testdata.47198 2023-12-01T06:28:27.943 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:27.943 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj74 testdata.47198 2023-12-01T06:28:29.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:29.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj75 testdata.47198 2023-12-01T06:28:31.076 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:31.077 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj76 testdata.47198 2023-12-01T06:28:32.632 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:32.632 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj77 testdata.47198 2023-12-01T06:28:34.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:34.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj78 testdata.47198 2023-12-01T06:28:35.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:35.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj79 testdata.47198 2023-12-01T06:28:37.210 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:37.211 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj80 testdata.47198 2023-12-01T06:28:38.942 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:38.943 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj81 testdata.47198 2023-12-01T06:28:40.416 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:40.416 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj82 testdata.47198 2023-12-01T06:28:41.949 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:41.950 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj83 testdata.47198 2023-12-01T06:28:43.487 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:43.487 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj84 testdata.47198 2023-12-01T06:28:45.111 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:45.111 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj85 testdata.47198 2023-12-01T06:28:46.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:46.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj86 testdata.47198 2023-12-01T06:28:48.367 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:48.367 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj87 testdata.47198 2023-12-01T06:28:49.957 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:49.957 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj88 testdata.47198 2023-12-01T06:28:51.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:51.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj89 testdata.47198 2023-12-01T06:28:53.072 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:53.072 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj90 testdata.47198 2023-12-01T06:28:54.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:54.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj91 testdata.47198 2023-12-01T06:28:56.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:56.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj92 testdata.47198 2023-12-01T06:28:57.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:57.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj93 testdata.47198 2023-12-01T06:28:59.308 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:28:59.309 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj94 testdata.47198 2023-12-01T06:29:00.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:29:00.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj95 testdata.47198 2023-12-01T06:29:02.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:29:02.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj96 testdata.47198 2023-12-01T06:29:03.963 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:29:03.963 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj97 testdata.47198 2023-12-01T06:29:05.568 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:29:05.568 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj98 testdata.47198 2023-12-01T06:29:07.190 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:29:07.191 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj99 testdata.47198 2023-12-01T06:29:08.864 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2023-12-01T06:29:08.865 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj100 testdata.47198 2023-12-01T06:29:10.763 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:72: TEST_recovery_scrub_1: rm -f testdata.47198 2023-12-01T06:29:10.773 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:74: TEST_recovery_scrub_1: ceph osd pool set test size 4 2023-12-01T06:29:11.185 INFO:tasks.workunit.client.0.smithi170.stderr:set pool 1 size to 4 2023-12-01T06:29:11.204 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:77: TEST_recovery_scrub_1: set -o pipefail 2023-12-01T06:29:11.204 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:78: TEST_recovery_scrub_1: count=0 2023-12-01T06:29:11.205 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2023-12-01T06:29:11.205 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2023-12-01T06:29:11.205 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2023-12-01T06:29:11.206 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2023-12-01T06:29:11.529 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:29:11.545 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:86: TEST_recovery_scrub_1: sleep 2 2023-12-01T06:29:13.546 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:87: TEST_recovery_scrub_1: test 0 -eq 10 2023-12-01T06:29:13.548 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: expr 0 + 1 2023-12-01T06:29:13.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: count=1 2023-12-01T06:29:13.550 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2023-12-01T06:29:13.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2023-12-01T06:29:13.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2023-12-01T06:29:13.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2023-12-01T06:29:13.848 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:29:13.866 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:86: TEST_recovery_scrub_1: sleep 2 2023-12-01T06:29:15.867 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:87: TEST_recovery_scrub_1: test 1 -eq 10 2023-12-01T06:29:15.868 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: expr 1 + 1 2023-12-01T06:29:15.870 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: count=2 2023-12-01T06:29:15.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2023-12-01T06:29:15.872 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2023-12-01T06:29:15.872 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2023-12-01T06:29:15.872 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2023-12-01T06:29:16.166 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:29:16.184 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:86: TEST_recovery_scrub_1: sleep 2 2023-12-01T06:29:18.185 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:87: TEST_recovery_scrub_1: test 2 -eq 10 2023-12-01T06:29:18.186 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: expr 2 + 1 2023-12-01T06:29:18.188 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: count=3 2023-12-01T06:29:18.189 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2023-12-01T06:29:18.190 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2023-12-01T06:29:18.190 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2023-12-01T06:29:18.190 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2023-12-01T06:29:18.482 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:29:18.499 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:84: TEST_recovery_scrub_1: break 2023-12-01T06:29:18.499 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:94: TEST_recovery_scrub_1: set +o pipefail 2023-12-01T06:29:18.499 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:95: TEST_recovery_scrub_1: ceph pg dump pgs 2023-12-01T06:29:18.944 INFO:tasks.workunit.client.0.smithi170.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2023-12-01T06:29:18.945 INFO:tasks.workunit.client.0.smithi170.stdout:1.0 100 0 300 0 0 5242880000 0 0 1300 0 1300 active+recovering+undersized+degraded+remapped 2023-12-01T06:29:12.968622+0000 24'1300 27:1322 [1,0,2,3] 1 [1,0] 1 0'0 2023-12-01T06:26:29.321857+0000 0'0 2023-12-01T06:26:29.321857+0000 0 0 periodic scrub scheduled @ 2023-12-02T11:17:07.726957+0000 0 0 2023-12-01T06:29:18.945 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:29:18.945 INFO:tasks.workunit.client.0.smithi170.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2023-12-01T06:29:18.945 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:29:18.962 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:97: TEST_recovery_scrub_1: sleep 10 2023-12-01T06:29:28.964 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:99: TEST_recovery_scrub_1: kill_daemons td/osd-recovery-scrub 2023-12-01T06:29:28.965 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-12-01T06:29:28.965 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-12-01T06:29:28.965 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-12-01T06:29:28.966 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-12-01T06:29:28.966 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-12-01T06:29:34.287 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-12-01T06:29:34.287 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:101: TEST_recovery_scrub_1: declare -a err_strings 2023-12-01T06:29:34.287 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:102: TEST_recovery_scrub_1: err_strings[0]='not scheduling scrubs due to active recovery' 2023-12-01T06:29:34.288 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: expr 4 - 1 2023-12-01T06:29:34.289 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: seq 0 3 2023-12-01T06:29:34.290 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:29:34.290 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'not scheduling scrubs' td/osd-recovery-scrub/osd.0.log 2023-12-01T06:29:34.295 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:16.440+0000 7fbdccbb5640 15 osd.0 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.295 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:25.391+0000 7fbdccbb5640 15 osd.0 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.295 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:26.427+0000 7fbdccbb5640 15 osd.0 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.295 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:31.503+0000 7fbdccbb5640 15 osd.0 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.295 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:32.487+0000 7fbdccbb5640 15 osd.0 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.296 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:29:34.296 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'not scheduling scrubs' td/osd-recovery-scrub/osd.1.log 2023-12-01T06:29:34.313 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:14.896+0000 7f2611f9a640 15 osd.1 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.313 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:17.964+0000 7f2611f9a640 15 osd.1 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.313 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:18.980+0000 7f2611f9a640 15 osd.1 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.313 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:20.919+0000 7f2611f9a640 15 osd.1 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.313 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:27.987+0000 7f2611f9a640 15 osd.1 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.313 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:32.011+0000 7f2611f9a640 15 osd.1 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:29:34.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'not scheduling scrubs' td/osd-recovery-scrub/osd.2.log 2023-12-01T06:29:34.316 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:22.751+0000 7f62bba1b640 15 osd.2 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.316 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:23.747+0000 7f62bba1b640 15 osd.2 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.316 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:25.775+0000 7f62bba1b640 15 osd.2 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.317 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:29.791+0000 7f62bba1b640 15 osd.2 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.317 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:30.843+0000 7f62bba1b640 15 osd.2 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:29:34.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'not scheduling scrubs' td/osd-recovery-scrub/osd.3.log 2023-12-01T06:29:34.320 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:17.632+0000 7f3de457a640 15 osd.3 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.320 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:18.632+0000 7f3de457a640 15 osd.3 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.320 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:20.623+0000 7f3de457a640 15 osd.3 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.320 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:22.619+0000 7f3de457a640 15 osd.3 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.320 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:26.583+0000 7f3de457a640 15 osd.3 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.320 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:27.627+0000 7f3de457a640 15 osd.3 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.320 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:31.591+0000 7f3de457a640 15 osd.3 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.320 INFO:tasks.workunit.client.0.smithi170.stdout:2023-12-01T06:29:32.599+0000 7f3de457a640 15 osd.3 osd-scrub:restrictions_on_scrubbing: not scheduling scrubs due to active recovery 2023-12-01T06:29:34.321 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:108: TEST_recovery_scrub_1: for err_string in "${err_strings[@]}" 2023-12-01T06:29:34.321 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:110: TEST_recovery_scrub_1: found=false 2023-12-01T06:29:34.321 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:111: TEST_recovery_scrub_1: count=0 2023-12-01T06:29:34.321 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: expr 4 - 1 2023-12-01T06:29:34.322 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: seq 0 3 2023-12-01T06:29:34.323 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:29:34.323 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'not scheduling scrubs due to active recovery' td/osd-recovery-scrub/osd.0.log 2023-12-01T06:29:34.325 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2023-12-01T06:29:34.326 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 0 + 1 2023-12-01T06:29:34.327 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=1 2023-12-01T06:29:34.327 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:29:34.327 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'not scheduling scrubs due to active recovery' td/osd-recovery-scrub/osd.1.log 2023-12-01T06:29:34.342 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2023-12-01T06:29:34.342 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 1 + 1 2023-12-01T06:29:34.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=2 2023-12-01T06:29:34.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:29:34.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'not scheduling scrubs due to active recovery' td/osd-recovery-scrub/osd.2.log 2023-12-01T06:29:34.347 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2023-12-01T06:29:34.348 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 2 + 1 2023-12-01T06:29:34.349 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=3 2023-12-01T06:29:34.349 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:29:34.349 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'not scheduling scrubs due to active recovery' td/osd-recovery-scrub/osd.3.log 2023-12-01T06:29:34.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2023-12-01T06:29:34.352 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 3 + 1 2023-12-01T06:29:34.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=4 2023-12-01T06:29:34.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:120: TEST_recovery_scrub_1: '[' true = false ']' 2023-12-01T06:29:34.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:124: TEST_recovery_scrub_1: '[' 4 -eq 4 ']' 2023-12-01T06:29:34.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:127: TEST_recovery_scrub_1: teardown td/osd-recovery-scrub 2023-12-01T06:29:34.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-recovery-scrub 2023-12-01T06:29:34.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-12-01T06:29:34.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-recovery-scrub KILL 2023-12-01T06:29:34.354 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-12-01T06:29:34.354 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-12-01T06:29:34.355 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-12-01T06:29:34.355 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-12-01T06:29:34.355 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-12-01T06:29:34.361 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-12-01T06:29:34.362 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-12-01T06:29:34.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-12-01T06:29:34.362 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-12-01T06:29:34.364 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-12-01T06:29:34.364 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-12-01T06:29:34.364 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-12-01T06:29:34.365 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:29:34.365 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-12-01T06:29:34.365 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-12-01T06:29:34.365 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:29:34.365 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-12-01T06:29:34.366 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-12-01T06:29:34.367 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-recovery-scrub 2023-12-01T06:29:34.391 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-12-01T06:29:34.391 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:29:34.391 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:29:34.391 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.47198 2023-12-01T06:29:34.392 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-12-01T06:29:34.392 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-12-01T06:29:34.392 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:129: TEST_recovery_scrub_1: '[' 0 '!=' 0 ']' 2023-12-01T06:29:34.392 INFO:tasks.workunit.client.0.smithi170.stdout:TEST PASSED 2023-12-01T06:29:34.394 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:135: TEST_recovery_scrub_1: echo 'TEST PASSED' 2023-12-01T06:29:34.394 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:136: TEST_recovery_scrub_1: return 0 2023-12-01T06:29:34.394 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:31: run: for func in $funcs 2023-12-01T06:29:34.394 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:32: run: TEST_recovery_scrub_2 td/osd-recovery-scrub 2023-12-01T06:29:34.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:234: TEST_recovery_scrub_2: local dir=td/osd-recovery-scrub 2023-12-01T06:29:34.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:235: TEST_recovery_scrub_2: local poolname=test 2023-12-01T06:29:34.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:237: TEST_recovery_scrub_2: TESTDATA=testdata.47198 2023-12-01T06:29:34.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:238: TEST_recovery_scrub_2: OSDS=8 2023-12-01T06:29:34.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:239: TEST_recovery_scrub_2: PGS=32 2023-12-01T06:29:34.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:240: TEST_recovery_scrub_2: OBJECTS=40 2023-12-01T06:29:34.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:242: TEST_recovery_scrub_2: setup td/osd-recovery-scrub 2023-12-01T06:29:34.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-recovery-scrub 2023-12-01T06:29:34.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-recovery-scrub 2023-12-01T06:29:34.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-recovery-scrub 2023-12-01T06:29:34.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-12-01T06:29:34.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-recovery-scrub KILL 2023-12-01T06:29:34.395 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-12-01T06:29:34.396 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-12-01T06:29:34.396 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-12-01T06:29:34.396 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-12-01T06:29:34.396 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-12-01T06:29:34.397 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-12-01T06:29:34.397 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-12-01T06:29:34.397 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-12-01T06:29:34.397 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-12-01T06:29:34.397 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-12-01T06:29:34.397 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-12-01T06:29:34.397 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-12-01T06:29:34.398 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:29:34.398 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-12-01T06:29:34.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-12-01T06:29:34.399 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:29:34.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-12-01T06:29:34.400 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-12-01T06:29:34.400 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-recovery-scrub 2023-12-01T06:29:34.401 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-12-01T06:29:34.402 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:29:34.402 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:29:34.402 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.47198 2023-12-01T06:29:34.402 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-12-01T06:29:34.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-12-01T06:29:34.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-recovery-scrub 2023-12-01T06:29:34.404 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2023-12-01T06:29:34.404 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:29:34.404 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:29:34.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.47198 2023-12-01T06:29:34.406 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2023-12-01T06:29:34.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2023-12-01T06:29:34.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2023-12-01T06:29:34.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-recovery-scrub 1' TERM HUP INT 2023-12-01T06:29:34.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:243: TEST_recovery_scrub_2: run_mon td/osd-recovery-scrub a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2023-12-01T06:29:34.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-recovery-scrub 2023-12-01T06:29:34.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2023-12-01T06:29:34.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2023-12-01T06:29:34.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2023-12-01T06:29:34.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-recovery-scrub/a 2023-12-01T06:29:34.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-recovery-scrub/a --run-dir=td/osd-recovery-scrub --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2023-12-01T06:29:34.778 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2023-12-01T06:29:34.778 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:29:34.778 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:29:34.779 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:29:34.779 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:29:34.779 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:29:34.779 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:29:34.780 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-recovery-scrub/a '--log-file=td/osd-recovery-scrub/$name.log' '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --mon-cluster-log-file=td/osd-recovery-scrub/log --run-dir=td/osd-recovery-scrub '--pid-file=td/osd-recovery-scrub/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2023-12-01T06:29:34.982 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2023-12-01T06:29:34.983 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2023-12-01T06:29:34.983 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2023-12-01T06:29:34.983 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2023-12-01T06:29:34.983 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2023-12-01T06:29:34.984 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2023-12-01T06:29:34.984 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2023-12-01T06:29:34.984 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-12-01T06:29:34.984 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-12-01T06:29:34.984 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:29:34.984 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:29:34.984 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:29:34.985 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.47198/ceph-mon.a.asok 2023-12-01T06:29:34.985 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2023-12-01T06:29:34.985 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.47198/ceph-mon.a.asok config get fsid 2023-12-01T06:29:35.101 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2023-12-01T06:29:35.101 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2023-12-01T06:29:35.101 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2023-12-01T06:29:35.101 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2023-12-01T06:29:35.102 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2023-12-01T06:29:35.102 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2023-12-01T06:29:35.102 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-12-01T06:29:35.102 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-12-01T06:29:35.103 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:29:35.103 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:29:35.103 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:29:35.103 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.47198/ceph-mon.a.asok 2023-12-01T06:29:35.103 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2023-12-01T06:29:35.103 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.47198/ceph-mon.a.asok config get mon_host 2023-12-01T06:29:35.213 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:244: TEST_recovery_scrub_2: run_mgr td/osd-recovery-scrub x 2023-12-01T06:29:35.213 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-recovery-scrub 2023-12-01T06:29:35.213 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2023-12-01T06:29:35.214 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2023-12-01T06:29:35.214 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2023-12-01T06:29:35.214 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-recovery-scrub/x 2023-12-01T06:29:35.214 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2023-12-01T06:29:36.080 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2023-12-01T06:29:36.080 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:29:36.080 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:29:36.081 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:29:36.081 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:29:36.081 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:29:36.081 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:29:36.082 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-12-01T06:29:36.083 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-recovery-scrub/x '--log-file=td/osd-recovery-scrub/$name.log' '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --run-dir=td/osd-recovery-scrub '--pid-file=td/osd-recovery-scrub/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-12-01T06:29:36.117 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:245: TEST_recovery_scrub_2: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 ' 2023-12-01T06:29:36.117 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:246: TEST_recovery_scrub_2: ceph_osd_args+='--osd_scrub_backoff_ratio=0 ' 2023-12-01T06:29:36.117 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:247: TEST_recovery_scrub_2: ceph_osd_args+='--osd_stats_update_period_not_scrubbing=3 ' 2023-12-01T06:29:36.117 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:248: TEST_recovery_scrub_2: ceph_osd_args+=--osd_stats_update_period_scrubbing=2 2023-12-01T06:29:36.117 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:249: TEST_recovery_scrub_2: expr 8 - 1 2023-12-01T06:29:36.118 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:249: TEST_recovery_scrub_2: seq 0 7 2023-12-01T06:29:36.121 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:249: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:29:36.121 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:251: TEST_recovery_scrub_2: run_osd td/osd-recovery-scrub 0 --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:29:36.121 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2023-12-01T06:29:36.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:29:36.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2023-12-01T06:29:36.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:29:36.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/0 2023-12-01T06:29:36.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2023-12-01T06:29:36.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:29:36.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:29:36.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:29:36.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/0' 2023-12-01T06:29:36.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/0/journal' 2023-12-01T06:29:36.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:29:36.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:29:36.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:29:36.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:29:36.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:29:36.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:29:36.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:29:36.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2023-12-01T06:29:36.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/0 2023-12-01T06:29:36.124 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:29:36.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=715afcdd-6484-4933-bdea-1f03611d0a00 2023-12-01T06:29:36.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 715afcdd-6484-4933-bdea-1f03611d0a00' 2023-12-01T06:29:36.125 INFO:tasks.workunit.client.0.smithi170.stdout:add osd0 715afcdd-6484-4933-bdea-1f03611d0a00 2023-12-01T06:29:36.125 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:29:36.141 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBQfWllPmqLCBAAAmaGhYmuDTs4nijh64eRtg== 2023-12-01T06:29:36.142 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBQfWllPmqLCBAAAmaGhYmuDTs4nijh64eRtg=="}' 2023-12-01T06:29:36.142 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 715afcdd-6484-4933-bdea-1f03611d0a00 -i td/osd-recovery-scrub/0/new.json 2023-12-01T06:29:36.401 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:29:36.417 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/0/new.json 2023-12-01T06:29:36.418 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/0 --osd-journal=td/osd-recovery-scrub/0/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQBQfWllPmqLCBAAAmaGhYmuDTs4nijh64eRtg== --osd-uuid 715afcdd-6484-4933-bdea-1f03611d0a00 2023-12-01T06:29:36.457 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:36.459+0000 7ff9ccfce640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:36.459 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:36.459+0000 7ff9ccfce640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:36.460 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:36.459+0000 7ff9ccfce640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:36.460 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:36.459+0000 7ff9ccfce640 -1 bluestore(td/osd-recovery-scrub/0/block) _read_bdev_label failed to open td/osd-recovery-scrub/0/block: (2) No such file or directory 2023-12-01T06:29:36.461 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:36.459+0000 7ff9ccfce640 -1 bluestore(td/osd-recovery-scrub/0/block) _read_bdev_label failed to open td/osd-recovery-scrub/0/block: (2) No such file or directory 2023-12-01T06:29:36.461 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:36.459+0000 7ff9ccfce640 -1 bluestore(td/osd-recovery-scrub/0/block) _read_bdev_label failed to open td/osd-recovery-scrub/0/block: (2) No such file or directory 2023-12-01T06:29:36.551 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:36.551+0000 7ff9ccfce640 -1 bluestore(td/osd-recovery-scrub/0) _read_fsid unparsable uuid 2023-12-01T06:29:42.405 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/0/keyring 2023-12-01T06:29:42.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:29:42.407 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd0 key to auth repository 2023-12-01T06:29:42.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2023-12-01T06:29:42.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:29:42.787 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.0 2023-12-01T06:29:42.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2023-12-01T06:29:42.788 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/0 --osd-journal=td/osd-recovery-scrub/0/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:29:42.788 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:29:42.788 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:29:42.788 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:29:42.826 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:42.827+0000 7fe487971640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:42.828 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:42.827+0000 7fe487971640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:42.829 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:42.831+0000 7fe487971640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:43.101 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2023-12-01T06:29:43.101 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:29:43.102 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2023-12-01T06:29:43.102 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:29:43.102 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:29:43.102 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:29:43.102 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:29:43.103 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:29:43.103 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:29:43.103 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:29:43.362 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:43.363+0000 7fe487971640 -1 Falling back to public interface 2023-12-01T06:29:43.417 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:29:44.417 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:29:44.418 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:29:44.418 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:29:44.419 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:29:44.419 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:29:44.419 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:29:44.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:29:45.721 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:29:45.721 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:29:45.721 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:29:45.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:29:45.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:29:45.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:29:45.846 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:45.847+0000 7fe487971640 -1 osd.0 0 log_to_monitors true 2023-12-01T06:29:46.034 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:29:47.035 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:29:47.035 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:29:47.036 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:29:47.036 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:29:47.036 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:29:47.036 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:29:47.348 INFO:tasks.workunit.client.0.smithi170.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/890317039,v1:127.0.0.1:6803/890317039] [v2:127.0.0.1:6804/890317039,v1:127.0.0.1:6805/890317039] exists,up 715afcdd-6484-4933-bdea-1f03611d0a00 2023-12-01T06:29:47.350 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:29:47.350 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:29:47.350 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:29:47.350 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:249: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:29:47.350 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:251: TEST_recovery_scrub_2: run_osd td/osd-recovery-scrub 1 --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/1 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/1' 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/1/journal' 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2023-12-01T06:29:47.351 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:29:47.352 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:29:47.352 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:29:47.352 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:29:47.352 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:29:47.352 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:29:47.353 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:29:47.353 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:29:47.353 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:29:47.353 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:29:47.353 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:29:47.353 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2023-12-01T06:29:47.353 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2023-12-01T06:29:47.353 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:29:47.353 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:29:47.353 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:29:47.353 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:29:47.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:29:47.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2023-12-01T06:29:47.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/1 2023-12-01T06:29:47.354 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:29:47.354 INFO:tasks.workunit.client.0.smithi170.stdout:add osd1 b66110e6-37f8-4496-9990-63b474834b84 2023-12-01T06:29:47.355 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b66110e6-37f8-4496-9990-63b474834b84 2023-12-01T06:29:47.355 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 b66110e6-37f8-4496-9990-63b474834b84' 2023-12-01T06:29:47.355 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:29:47.377 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBbfWllZragFhAAlgEm2DwlBCLJqZ7BzEzI8g== 2023-12-01T06:29:47.377 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBbfWllZragFhAAlgEm2DwlBCLJqZ7BzEzI8g=="}' 2023-12-01T06:29:47.377 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b66110e6-37f8-4496-9990-63b474834b84 -i td/osd-recovery-scrub/1/new.json 2023-12-01T06:29:47.732 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:29:47.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/1/new.json 2023-12-01T06:29:47.751 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/1 --osd-journal=td/osd-recovery-scrub/1/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQBbfWllZragFhAAlgEm2DwlBCLJqZ7BzEzI8g== --osd-uuid b66110e6-37f8-4496-9990-63b474834b84 2023-12-01T06:29:47.787 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:47.787+0000 7f480d73c640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:47.817 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:47.815+0000 7f480d73c640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:47.818 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:47.819+0000 7f480d73c640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:47.819 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:47.819+0000 7f480d73c640 -1 bluestore(td/osd-recovery-scrub/1/block) _read_bdev_label failed to open td/osd-recovery-scrub/1/block: (2) No such file or directory 2023-12-01T06:29:47.819 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:47.819+0000 7f480d73c640 -1 bluestore(td/osd-recovery-scrub/1/block) _read_bdev_label failed to open td/osd-recovery-scrub/1/block: (2) No such file or directory 2023-12-01T06:29:47.819 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:47.819+0000 7f480d73c640 -1 bluestore(td/osd-recovery-scrub/1/block) _read_bdev_label failed to open td/osd-recovery-scrub/1/block: (2) No such file or directory 2023-12-01T06:29:47.891 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:47.891+0000 7f480d73c640 -1 bluestore(td/osd-recovery-scrub/1) _read_fsid unparsable uuid 2023-12-01T06:29:54.274 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/1/keyring 2023-12-01T06:29:54.275 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:29:54.277 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd1 key to auth repository 2023-12-01T06:29:54.277 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2023-12-01T06:29:54.277 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:29:54.656 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.1 2023-12-01T06:29:54.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2023-12-01T06:29:54.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/1 --osd-journal=td/osd-recovery-scrub/1/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:29:54.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:29:54.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:29:54.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:29:54.692 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:54.691+0000 7f367eb0d640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:54.715 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:54.715+0000 7f367eb0d640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:54.716 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:54.719+0000 7f367eb0d640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:54.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2023-12-01T06:29:54.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:29:54.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2023-12-01T06:29:54.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:29:54.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:29:54.979 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:29:54.979 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:29:54.980 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:29:54.980 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:29:54.980 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:29:55.279 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:55.279+0000 7f367eb0d640 -1 Falling back to public interface 2023-12-01T06:29:55.341 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:29:56.342 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:29:56.342 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:29:56.342 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:29:56.343 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:29:56.343 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:29:56.343 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:29:56.655 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:29:57.656 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:29:57.656 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:29:57.657 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:29:57.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:29:57.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:29:57.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:29:57.802 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:57.803+0000 7f367eb0d640 -1 osd.1 0 log_to_monitors true 2023-12-01T06:29:58.027 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:29:59.029 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:29:59.029 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:29:59.029 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:29:59.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:29:59.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:29:59.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:29:59.400 INFO:tasks.workunit.client.0.smithi170.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2762943556,v1:127.0.0.1:6811/2762943556] [v2:127.0.0.1:6812/2762943556,v1:127.0.0.1:6813/2762943556] exists,up b66110e6-37f8-4496-9990-63b474834b84 2023-12-01T06:29:59.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:29:59.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:29:59.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:29:59.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:249: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:29:59.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:251: TEST_recovery_scrub_2: run_osd td/osd-recovery-scrub 2 --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:29:59.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2023-12-01T06:29:59.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:29:59.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2023-12-01T06:29:59.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:29:59.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/2 2023-12-01T06:29:59.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2023-12-01T06:29:59.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:29:59.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:29:59.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:29:59.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/2' 2023-12-01T06:29:59.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/2/journal' 2023-12-01T06:29:59.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:29:59.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:29:59.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2023-12-01T06:29:59.404 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:29:59.404 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:29:59.404 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:29:59.405 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:29:59.405 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:29:59.405 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:29:59.405 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:29:59.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:29:59.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:29:59.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:29:59.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:29:59.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2023-12-01T06:29:59.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2023-12-01T06:29:59.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:29:59.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:29:59.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:29:59.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:29:59.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:29:59.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2023-12-01T06:29:59.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/2 2023-12-01T06:29:59.408 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:29:59.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=44c9cd21-8277-4651-8d3a-14932146d3b6 2023-12-01T06:29:59.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 44c9cd21-8277-4651-8d3a-14932146d3b6' 2023-12-01T06:29:59.408 INFO:tasks.workunit.client.0.smithi170.stdout:add osd2 44c9cd21-8277-4651-8d3a-14932146d3b6 2023-12-01T06:29:59.408 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:29:59.425 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBnfWllVx5/GRAARlb8aBGpF750qPtjJ6CArg== 2023-12-01T06:29:59.425 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBnfWllVx5/GRAARlb8aBGpF750qPtjJ6CArg=="}' 2023-12-01T06:29:59.425 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 44c9cd21-8277-4651-8d3a-14932146d3b6 -i td/osd-recovery-scrub/2/new.json 2023-12-01T06:29:59.765 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:29:59.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/2/new.json 2023-12-01T06:29:59.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/2 --osd-journal=td/osd-recovery-scrub/2/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQBnfWllVx5/GRAARlb8aBGpF750qPtjJ6CArg== --osd-uuid 44c9cd21-8277-4651-8d3a-14932146d3b6 2023-12-01T06:29:59.817 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:59.819+0000 7f810775f640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:59.851 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:59.851+0000 7f810775f640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:59.852 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:59.855+0000 7f810775f640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:29:59.853 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:59.855+0000 7f810775f640 -1 bluestore(td/osd-recovery-scrub/2/block) _read_bdev_label failed to open td/osd-recovery-scrub/2/block: (2) No such file or directory 2023-12-01T06:29:59.853 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:59.855+0000 7f810775f640 -1 bluestore(td/osd-recovery-scrub/2/block) _read_bdev_label failed to open td/osd-recovery-scrub/2/block: (2) No such file or directory 2023-12-01T06:29:59.853 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:59.855+0000 7f810775f640 -1 bluestore(td/osd-recovery-scrub/2/block) _read_bdev_label failed to open td/osd-recovery-scrub/2/block: (2) No such file or directory 2023-12-01T06:29:59.926 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:29:59.927+0000 7f810775f640 -1 bluestore(td/osd-recovery-scrub/2) _read_fsid unparsable uuid 2023-12-01T06:30:06.623 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/2/keyring 2023-12-01T06:30:06.624 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:30:06.624 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2023-12-01T06:30:06.625 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd2 key to auth repository 2023-12-01T06:30:06.625 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:30:07.010 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.2 2023-12-01T06:30:07.010 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2023-12-01T06:30:07.010 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/2 --osd-journal=td/osd-recovery-scrub/2/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:30:07.010 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:30:07.011 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:30:07.011 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:30:07.047 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:07.047+0000 7f13b574b640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:07.068 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:07.067+0000 7f13b574b640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:07.069 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:07.071+0000 7f13b574b640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:07.330 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2023-12-01T06:30:07.330 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:30:07.330 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2023-12-01T06:30:07.330 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:30:07.330 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:30:07.330 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:07.330 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:30:07.331 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:30:07.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:07.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:30:07.626 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:07.627+0000 7f13b574b640 -1 Falling back to public interface 2023-12-01T06:30:07.652 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:08.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:08.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:08.653 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:30:08.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:30:08.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:08.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:30:08.957 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:09.959 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:09.959 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:09.959 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:30:09.960 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:30:09.960 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:09.960 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:30:10.265 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:10.267+0000 7f13b574b640 -1 osd.2 0 log_to_monitors true 2023-12-01T06:30:10.270 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:11.272 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:11.272 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:11.272 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:30:11.272 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:30:11.272 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:11.273 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:30:11.596 INFO:tasks.workunit.client.0.smithi170.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/805742994,v1:127.0.0.1:6819/805742994] [v2:127.0.0.1:6820/805742994,v1:127.0.0.1:6821/805742994] exists,up 44c9cd21-8277-4651-8d3a-14932146d3b6 2023-12-01T06:30:11.600 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:30:11.600 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:30:11.600 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:30:11.600 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:249: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:30:11.600 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:251: TEST_recovery_scrub_2: run_osd td/osd-recovery-scrub 3 --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:30:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2023-12-01T06:30:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:30:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2023-12-01T06:30:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:30:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/3 2023-12-01T06:30:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2023-12-01T06:30:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:30:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:30:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:30:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/3' 2023-12-01T06:30:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/3/journal' 2023-12-01T06:30:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:30:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:30:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2023-12-01T06:30:11.602 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:30:11.602 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:30:11.602 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:30:11.602 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:30:11.602 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:30:11.602 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:30:11.602 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:30:11.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:30:11.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:30:11.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:30:11.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:30:11.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2023-12-01T06:30:11.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2023-12-01T06:30:11.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:30:11.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:30:11.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:30:11.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:30:11.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:30:11.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2023-12-01T06:30:11.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/3 2023-12-01T06:30:11.603 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:30:11.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=95080be2-eb76-4cbd-b1b6-7c7147900077 2023-12-01T06:30:11.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 95080be2-eb76-4cbd-b1b6-7c7147900077' 2023-12-01T06:30:11.604 INFO:tasks.workunit.client.0.smithi170.stdout:add osd3 95080be2-eb76-4cbd-b1b6-7c7147900077 2023-12-01T06:30:11.604 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:30:11.623 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBzfWlld1dIJRAArjIXDGIh961BNzy0up94Rg== 2023-12-01T06:30:11.623 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBzfWlld1dIJRAArjIXDGIh961BNzy0up94Rg=="}' 2023-12-01T06:30:11.623 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 95080be2-eb76-4cbd-b1b6-7c7147900077 -i td/osd-recovery-scrub/3/new.json 2023-12-01T06:30:11.975 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:30:11.994 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/3/new.json 2023-12-01T06:30:11.995 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/3 --osd-journal=td/osd-recovery-scrub/3/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQBzfWlld1dIJRAArjIXDGIh961BNzy0up94Rg== --osd-uuid 95080be2-eb76-4cbd-b1b6-7c7147900077 2023-12-01T06:30:12.035 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:12.035+0000 7f672ec2f640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:12.059 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:12.059+0000 7f672ec2f640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:12.060 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:12.063+0000 7f672ec2f640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:12.061 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:12.063+0000 7f672ec2f640 -1 bluestore(td/osd-recovery-scrub/3/block) _read_bdev_label failed to open td/osd-recovery-scrub/3/block: (2) No such file or directory 2023-12-01T06:30:12.061 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:12.063+0000 7f672ec2f640 -1 bluestore(td/osd-recovery-scrub/3/block) _read_bdev_label failed to open td/osd-recovery-scrub/3/block: (2) No such file or directory 2023-12-01T06:30:12.061 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:12.063+0000 7f672ec2f640 -1 bluestore(td/osd-recovery-scrub/3/block) _read_bdev_label failed to open td/osd-recovery-scrub/3/block: (2) No such file or directory 2023-12-01T06:30:12.135 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:12.135+0000 7f672ec2f640 -1 bluestore(td/osd-recovery-scrub/3) _read_fsid unparsable uuid 2023-12-01T06:30:18.566 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/3/keyring 2023-12-01T06:30:18.566 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:30:18.567 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd3 key to auth repository 2023-12-01T06:30:18.567 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2023-12-01T06:30:18.567 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:30:18.955 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.3 2023-12-01T06:30:18.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2023-12-01T06:30:18.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/3 --osd-journal=td/osd-recovery-scrub/3/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:30:18.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:30:18.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:30:18.956 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:30:18.987 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:18.987+0000 7f39504ef640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:19.018 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:19.019+0000 7f39504ef640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:19.019 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:19.019+0000 7f39504ef640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:19.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2023-12-01T06:30:19.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:30:19.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2023-12-01T06:30:19.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:30:19.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:30:19.289 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:19.289 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:30:19.289 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:30:19.289 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:19.289 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2023-12-01T06:30:19.570 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:19.571+0000 7f39504ef640 -1 Falling back to public interface 2023-12-01T06:30:19.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:20.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:20.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:20.594 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:30:20.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:30:20.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:20.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2023-12-01T06:30:20.905 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:21.906 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:21.906 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:21.906 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:30:21.907 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:30:21.907 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:21.907 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2023-12-01T06:30:22.203 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:22.203+0000 7f39504ef640 -1 osd.3 0 log_to_monitors true 2023-12-01T06:30:22.211 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:23.213 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:23.213 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:23.213 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:30:23.213 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:30:23.214 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:23.214 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2023-12-01T06:30:23.543 INFO:tasks.workunit.client.0.smithi170.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3718093716,v1:127.0.0.1:6827/3718093716] [v2:127.0.0.1:6828/3718093716,v1:127.0.0.1:6829/3718093716] exists,up 95080be2-eb76-4cbd-b1b6-7c7147900077 2023-12-01T06:30:23.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:30:23.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:30:23.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:30:23.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:249: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:30:23.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:251: TEST_recovery_scrub_2: run_osd td/osd-recovery-scrub 4 --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:30:23.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2023-12-01T06:30:23.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:30:23.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=4 2023-12-01T06:30:23.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:30:23.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/4 2023-12-01T06:30:23.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2023-12-01T06:30:23.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:30:23.548 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:30:23.548 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:30:23.548 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/4' 2023-12-01T06:30:23.548 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/4/journal' 2023-12-01T06:30:23.548 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:30:23.548 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:30:23.548 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2023-12-01T06:30:23.548 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:30:23.548 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:30:23.548 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:30:23.548 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:30:23.548 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:30:23.548 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:30:23.548 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:30:23.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:30:23.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:30:23.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:30:23.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:30:23.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2023-12-01T06:30:23.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2023-12-01T06:30:23.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:30:23.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:30:23.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:30:23.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:30:23.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:30:23.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2023-12-01T06:30:23.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/4 2023-12-01T06:30:23.549 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:30:23.550 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=811987f0-9ad3-483c-b99f-7763ed8ded23 2023-12-01T06:30:23.550 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd4 811987f0-9ad3-483c-b99f-7763ed8ded23' 2023-12-01T06:30:23.550 INFO:tasks.workunit.client.0.smithi170.stdout:add osd4 811987f0-9ad3-483c-b99f-7763ed8ded23 2023-12-01T06:30:23.551 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:30:23.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB/fWll5ObLIRAAV7hH3b8g2Xuy2cruZGkNhQ== 2023-12-01T06:30:23.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB/fWll5ObLIRAAV7hH3b8g2Xuy2cruZGkNhQ=="}' 2023-12-01T06:30:23.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 811987f0-9ad3-483c-b99f-7763ed8ded23 -i td/osd-recovery-scrub/4/new.json 2023-12-01T06:30:23.934 INFO:tasks.workunit.client.0.smithi170.stdout:4 2023-12-01T06:30:23.953 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/4/new.json 2023-12-01T06:30:23.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 4 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/4 --osd-journal=td/osd-recovery-scrub/4/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQB/fWll5ObLIRAAV7hH3b8g2Xuy2cruZGkNhQ== --osd-uuid 811987f0-9ad3-483c-b99f-7763ed8ded23 2023-12-01T06:30:23.995 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:23.995+0000 7fd49c91f640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:24.018 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:24.019+0000 7fd49c91f640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:24.019 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:24.019+0000 7fd49c91f640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:24.020 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:24.019+0000 7fd49c91f640 -1 bluestore(td/osd-recovery-scrub/4/block) _read_bdev_label failed to open td/osd-recovery-scrub/4/block: (2) No such file or directory 2023-12-01T06:30:24.020 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:24.019+0000 7fd49c91f640 -1 bluestore(td/osd-recovery-scrub/4/block) _read_bdev_label failed to open td/osd-recovery-scrub/4/block: (2) No such file or directory 2023-12-01T06:30:24.020 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:24.019+0000 7fd49c91f640 -1 bluestore(td/osd-recovery-scrub/4/block) _read_bdev_label failed to open td/osd-recovery-scrub/4/block: (2) No such file or directory 2023-12-01T06:30:24.090 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:24.091+0000 7fd49c91f640 -1 bluestore(td/osd-recovery-scrub/4) _read_fsid unparsable uuid 2023-12-01T06:30:30.507 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/4/keyring 2023-12-01T06:30:30.507 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:30:30.507 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd4 key to auth repository 2023-12-01T06:30:30.508 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd4 key to auth repository 2023-12-01T06:30:30.508 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:30:30.900 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.4 2023-12-01T06:30:30.900 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.4 2023-12-01T06:30:30.900 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 4 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/4 --osd-journal=td/osd-recovery-scrub/4/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:30:30.900 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:30:30.900 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:30:30.900 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:30:30.936 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:30.939+0000 7fe6f64bb640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:30.970 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:30.971+0000 7fe6f64bb640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:30.972 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:30.971+0000 7fe6f64bb640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:31.238 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 4 2023-12-01T06:30:31.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:30:31.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=4 2023-12-01T06:30:31.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:30:31.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:30:31.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:31.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:30:31.239 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:30:31.240 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:31.240 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2023-12-01T06:30:31.526 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:31.527+0000 7fe6f64bb640 -1 Falling back to public interface 2023-12-01T06:30:31.571 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:32.572 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:32.572 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:32.573 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:30:32.573 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:30:32.573 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:32.573 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2023-12-01T06:30:32.885 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:33.886 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:33.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:33.887 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:30:33.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:30:33.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:33.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2023-12-01T06:30:34.066 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:34.067+0000 7fe6f64bb640 -1 osd.4 0 log_to_monitors true 2023-12-01T06:30:34.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:35.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:35.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:35.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:30:35.288 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:30:35.289 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:35.289 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2023-12-01T06:30:35.601 INFO:tasks.workunit.client.0.smithi170.stdout:osd.4 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2122915852,v1:127.0.0.1:6835/2122915852] [v2:127.0.0.1:6836/2122915852,v1:127.0.0.1:6837/2122915852] exists,up 811987f0-9ad3-483c-b99f-7763ed8ded23 2023-12-01T06:30:35.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:30:35.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:30:35.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:30:35.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:249: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:30:35.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:251: TEST_recovery_scrub_2: run_osd td/osd-recovery-scrub 5 --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:30:35.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2023-12-01T06:30:35.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:30:35.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=5 2023-12-01T06:30:35.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:30:35.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/5 2023-12-01T06:30:35.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2023-12-01T06:30:35.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:30:35.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:30:35.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:30:35.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/5' 2023-12-01T06:30:35.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/5/journal' 2023-12-01T06:30:35.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:30:35.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:30:35.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2023-12-01T06:30:35.605 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:30:35.605 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:30:35.605 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:30:35.606 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:30:35.606 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:30:35.606 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:30:35.606 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:30:35.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:30:35.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:30:35.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:30:35.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:30:35.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2023-12-01T06:30:35.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2023-12-01T06:30:35.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:30:35.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:30:35.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:30:35.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:30:35.607 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:30:35.607 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2023-12-01T06:30:35.607 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/5 2023-12-01T06:30:35.607 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:30:35.607 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=35985ce2-8b97-421f-adef-63ba4f5c3c79 2023-12-01T06:30:35.608 INFO:tasks.workunit.client.0.smithi170.stdout:add osd5 35985ce2-8b97-421f-adef-63ba4f5c3c79 2023-12-01T06:30:35.608 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd5 35985ce2-8b97-421f-adef-63ba4f5c3c79' 2023-12-01T06:30:35.608 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:30:35.627 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCLfWlltJCQJRAAc45tTZBQTwVXein6Xqs4yQ== 2023-12-01T06:30:35.627 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCLfWlltJCQJRAAc45tTZBQTwVXein6Xqs4yQ=="}' 2023-12-01T06:30:35.627 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 35985ce2-8b97-421f-adef-63ba4f5c3c79 -i td/osd-recovery-scrub/5/new.json 2023-12-01T06:30:36.003 INFO:tasks.workunit.client.0.smithi170.stdout:5 2023-12-01T06:30:36.021 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/5/new.json 2023-12-01T06:30:36.023 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 5 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/5 --osd-journal=td/osd-recovery-scrub/5/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQCLfWlltJCQJRAAc45tTZBQTwVXein6Xqs4yQ== --osd-uuid 35985ce2-8b97-421f-adef-63ba4f5c3c79 2023-12-01T06:30:36.063 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:36.063+0000 7fa15a2a9640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:36.065 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:36.067+0000 7fa15a2a9640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:36.066 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:36.067+0000 7fa15a2a9640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:36.067 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:36.067+0000 7fa15a2a9640 -1 bluestore(td/osd-recovery-scrub/5/block) _read_bdev_label failed to open td/osd-recovery-scrub/5/block: (2) No such file or directory 2023-12-01T06:30:36.067 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:36.067+0000 7fa15a2a9640 -1 bluestore(td/osd-recovery-scrub/5/block) _read_bdev_label failed to open td/osd-recovery-scrub/5/block: (2) No such file or directory 2023-12-01T06:30:36.067 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:36.067+0000 7fa15a2a9640 -1 bluestore(td/osd-recovery-scrub/5/block) _read_bdev_label failed to open td/osd-recovery-scrub/5/block: (2) No such file or directory 2023-12-01T06:30:36.187 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:36.187+0000 7fa15a2a9640 -1 bluestore(td/osd-recovery-scrub/5) _read_fsid unparsable uuid 2023-12-01T06:30:42.797 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/5/keyring 2023-12-01T06:30:42.797 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:30:42.799 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd5 key to auth repository 2023-12-01T06:30:42.799 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd5 key to auth repository 2023-12-01T06:30:42.799 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:30:43.169 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.5 2023-12-01T06:30:43.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.5 2023-12-01T06:30:43.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 5 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/5 --osd-journal=td/osd-recovery-scrub/5/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:30:43.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:30:43.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:30:43.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:30:43.207 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:43.207+0000 7f197a55c640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:43.215 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:43.215+0000 7f197a55c640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:43.216 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:43.219+0000 7f197a55c640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:43.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 5 2023-12-01T06:30:43.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:30:43.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2023-12-01T06:30:43.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:30:43.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:30:43.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:43.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:30:43.462 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:30:43.463 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:43.463 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2023-12-01T06:30:43.773 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:43.773 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:43.775+0000 7f197a55c640 -1 Falling back to public interface 2023-12-01T06:30:44.774 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:44.774 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:44.774 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:30:44.775 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:30:44.775 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:44.775 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2023-12-01T06:30:45.085 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:46.087 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:46.087 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:46.087 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:30:46.088 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:30:46.088 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:46.088 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2023-12-01T06:30:46.290 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:46.291+0000 7f197a55c640 -1 osd.5 0 log_to_monitors true 2023-12-01T06:30:46.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:47.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:47.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:47.407 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:30:47.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:30:47.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:47.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2023-12-01T06:30:47.774 INFO:tasks.workunit.client.0.smithi170.stdout:osd.5 up in weight 1 up_from 30 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/2155388767,v1:127.0.0.1:6843/2155388767] [v2:127.0.0.1:6844/2155388767,v1:127.0.0.1:6845/2155388767] exists,up 35985ce2-8b97-421f-adef-63ba4f5c3c79 2023-12-01T06:30:47.780 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:30:47.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:30:47.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:30:47.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:249: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:30:47.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:251: TEST_recovery_scrub_2: run_osd td/osd-recovery-scrub 6 --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:30:47.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2023-12-01T06:30:47.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:30:47.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=6 2023-12-01T06:30:47.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:30:47.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/6 2023-12-01T06:30:47.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2023-12-01T06:30:47.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:30:47.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:30:47.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/6' 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/6/journal' 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:30:47.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:30:47.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:30:47.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2023-12-01T06:30:47.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2023-12-01T06:30:47.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:30:47.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:30:47.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:30:47.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:30:47.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:30:47.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2023-12-01T06:30:47.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/6 2023-12-01T06:30:47.783 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:30:47.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=4a05ab96-ccc7-40ba-9ace-a6bdb20e94da 2023-12-01T06:30:47.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd6 4a05ab96-ccc7-40ba-9ace-a6bdb20e94da' 2023-12-01T06:30:47.784 INFO:tasks.workunit.client.0.smithi170.stdout:add osd6 4a05ab96-ccc7-40ba-9ace-a6bdb20e94da 2023-12-01T06:30:47.784 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:30:47.796 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCXfWllWu+kLxAAtpT204DLnzwrOpoNPNPrMA== 2023-12-01T06:30:47.796 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCXfWllWu+kLxAAtpT204DLnzwrOpoNPNPrMA=="}' 2023-12-01T06:30:47.796 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 4a05ab96-ccc7-40ba-9ace-a6bdb20e94da -i td/osd-recovery-scrub/6/new.json 2023-12-01T06:30:48.158 INFO:tasks.workunit.client.0.smithi170.stdout:6 2023-12-01T06:30:48.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/6/new.json 2023-12-01T06:30:48.178 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 6 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/6 --osd-journal=td/osd-recovery-scrub/6/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQCXfWllWu+kLxAAtpT204DLnzwrOpoNPNPrMA== --osd-uuid 4a05ab96-ccc7-40ba-9ace-a6bdb20e94da 2023-12-01T06:30:48.213 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:48.215+0000 7f8042a6b640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:48.243 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:48.243+0000 7f8042a6b640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:48.244 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:48.247+0000 7f8042a6b640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:48.245 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:48.247+0000 7f8042a6b640 -1 bluestore(td/osd-recovery-scrub/6/block) _read_bdev_label failed to open td/osd-recovery-scrub/6/block: (2) No such file or directory 2023-12-01T06:30:48.245 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:48.247+0000 7f8042a6b640 -1 bluestore(td/osd-recovery-scrub/6/block) _read_bdev_label failed to open td/osd-recovery-scrub/6/block: (2) No such file or directory 2023-12-01T06:30:48.245 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:48.247+0000 7f8042a6b640 -1 bluestore(td/osd-recovery-scrub/6/block) _read_bdev_label failed to open td/osd-recovery-scrub/6/block: (2) No such file or directory 2023-12-01T06:30:48.305 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:48.307+0000 7f8042a6b640 -1 bluestore(td/osd-recovery-scrub/6) _read_fsid unparsable uuid 2023-12-01T06:30:54.933 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/6/keyring 2023-12-01T06:30:54.934 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:30:54.935 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd6 key to auth repository 2023-12-01T06:30:54.936 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd6 key to auth repository 2023-12-01T06:30:54.936 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:30:55.323 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.6 2023-12-01T06:30:55.324 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.6 2023-12-01T06:30:55.324 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:30:55.324 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 6 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/6 --osd-journal=td/osd-recovery-scrub/6/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:30:55.324 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:30:55.324 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:30:55.361 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:55.363+0000 7f13324ae640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:55.364 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:55.367+0000 7f13324ae640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:55.365 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:55.367+0000 7f13324ae640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:30:55.614 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 6 2023-12-01T06:30:55.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:30:55.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=6 2023-12-01T06:30:55.615 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:30:55.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:30:55.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:30:55.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:55.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:30:55.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:55.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2023-12-01T06:30:55.920 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:55.925 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:55.927+0000 7f13324ae640 -1 Falling back to public interface 2023-12-01T06:30:56.922 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:56.922 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:56.922 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:30:56.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:30:56.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:56.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2023-12-01T06:30:57.224 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:58.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:58.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:58.226 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:30:58.226 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:30:58.226 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:58.226 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2023-12-01T06:30:58.473 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:30:58.474+0000 7f13324ae640 -1 osd.6 0 log_to_monitors true 2023-12-01T06:30:58.537 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:30:59.538 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:30:59.538 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:30:59.539 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:30:59.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:30:59.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:30:59.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2023-12-01T06:30:59.885 INFO:tasks.workunit.client.0.smithi170.stdout:osd.6 up in weight 1 up_from 35 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3797038562,v1:127.0.0.1:6851/3797038562] [v2:127.0.0.1:6852/3797038562,v1:127.0.0.1:6853/3797038562] exists,up 4a05ab96-ccc7-40ba-9ace-a6bdb20e94da 2023-12-01T06:30:59.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:30:59.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:30:59.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:30:59.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:249: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:30:59.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:251: TEST_recovery_scrub_2: run_osd td/osd-recovery-scrub 7 --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:30:59.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2023-12-01T06:30:59.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:30:59.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=7 2023-12-01T06:30:59.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:30:59.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/7 2023-12-01T06:30:59.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/7' 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/7/journal' 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:30:59.888 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:30:59.889 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' 2023-12-01T06:30:59.889 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:30:59.889 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:30:59.889 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:30:59.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2023-12-01T06:30:59.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2023-12-01T06:30:59.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:30:59.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:30:59.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:30:59.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:30:59.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:30:59.891 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2023-12-01T06:30:59.891 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/7 2023-12-01T06:30:59.891 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:30:59.891 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c96be879-56b0-4fb1-9ac7-773fd1bc72ee 2023-12-01T06:30:59.891 INFO:tasks.workunit.client.0.smithi170.stdout:add osd7 c96be879-56b0-4fb1-9ac7-773fd1bc72ee 2023-12-01T06:30:59.892 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd7 c96be879-56b0-4fb1-9ac7-773fd1bc72ee' 2023-12-01T06:30:59.892 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:30:59.908 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCjfWllK6lcNhAAPht8WJ2PyjIiMPYrD6txMA== 2023-12-01T06:30:59.908 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCjfWllK6lcNhAAPht8WJ2PyjIiMPYrD6txMA=="}' 2023-12-01T06:30:59.909 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c96be879-56b0-4fb1-9ac7-773fd1bc72ee -i td/osd-recovery-scrub/7/new.json 2023-12-01T06:31:00.320 INFO:tasks.workunit.client.0.smithi170.stdout:7 2023-12-01T06:31:00.338 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/7/new.json 2023-12-01T06:31:00.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 7 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/7 --osd-journal=td/osd-recovery-scrub/7/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQCjfWllK6lcNhAAPht8WJ2PyjIiMPYrD6txMA== --osd-uuid c96be879-56b0-4fb1-9ac7-773fd1bc72ee 2023-12-01T06:31:00.379 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:31:00.378+0000 7f5e1f9bb640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:31:00.417 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:31:00.418+0000 7f5e1f9bb640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:31:00.419 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:31:00.418+0000 7f5e1f9bb640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:31:00.420 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:31:00.422+0000 7f5e1f9bb640 -1 bluestore(td/osd-recovery-scrub/7/block) _read_bdev_label failed to open td/osd-recovery-scrub/7/block: (2) No such file or directory 2023-12-01T06:31:00.420 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:31:00.422+0000 7f5e1f9bb640 -1 bluestore(td/osd-recovery-scrub/7/block) _read_bdev_label failed to open td/osd-recovery-scrub/7/block: (2) No such file or directory 2023-12-01T06:31:00.420 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:31:00.422+0000 7f5e1f9bb640 -1 bluestore(td/osd-recovery-scrub/7/block) _read_bdev_label failed to open td/osd-recovery-scrub/7/block: (2) No such file or directory 2023-12-01T06:31:00.508 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:31:00.510+0000 7f5e1f9bb640 -1 bluestore(td/osd-recovery-scrub/7) _read_fsid unparsable uuid 2023-12-01T06:31:07.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/7/keyring 2023-12-01T06:31:07.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:31:07.655 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd7 key to auth repository 2023-12-01T06:31:07.656 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd7 key to auth repository 2023-12-01T06:31:07.656 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:31:08.029 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.7 2023-12-01T06:31:08.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.7 2023-12-01T06:31:08.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 7 --fsid=d894a706-157c-44dc-8499-81c8685b68f1 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/7 --osd-journal=td/osd-recovery-scrub/7/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.47198/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=true --osd_recovery_sleep=10 --osd-scrub-interval-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2023-12-01T06:31:08.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:31:08.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:31:08.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:31:08.066 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:31:08.066+0000 7f33fb1e1640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:31:08.094 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:31:08.094+0000 7f33fb1e1640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:31:08.095 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:31:08.094+0000 7f33fb1e1640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:31:08.357 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 7 2023-12-01T06:31:08.357 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:31:08.357 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=7 2023-12-01T06:31:08.358 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:31:08.358 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:31:08.358 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:31:08.358 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:31:08.358 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:31:08.358 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:31:08.358 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2023-12-01T06:31:08.638 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:31:08.638+0000 7f33fb1e1640 -1 Falling back to public interface 2023-12-01T06:31:08.677 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:31:09.678 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:31:09.678 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:31:09.678 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:31:09.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:31:09.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:31:09.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2023-12-01T06:31:09.992 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:31:10.993 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:31:10.994 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:31:10.994 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:31:10.994 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:31:10.994 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:31:10.994 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2023-12-01T06:31:11.222 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:31:11.222+0000 7f33fb1e1640 -1 osd.7 0 log_to_monitors true 2023-12-01T06:31:11.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:31:12.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:31:12.405 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:31:12.405 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:31:12.405 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:31:12.405 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:31:12.405 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2023-12-01T06:31:12.730 INFO:tasks.workunit.client.0.smithi170.stdout:osd.7 up in weight 1 up_from 40 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/117631179,v1:127.0.0.1:6859/117631179] [v2:127.0.0.1:6860/117631179,v1:127.0.0.1:6861/117631179] exists,up c96be879-56b0-4fb1-9ac7-773fd1bc72ee 2023-12-01T06:31:12.730 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:31:12.731 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:31:12.731 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:31:12.731 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:256: TEST_recovery_scrub_2: create_pool test 32 32 2023-12-01T06:31:12.731 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 32 32 2023-12-01T06:31:13.136 INFO:tasks.workunit.client.0.smithi170.stderr:pool 'test' created 2023-12-01T06:31:13.157 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2023-12-01T06:31:14.159 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:257: TEST_recovery_scrub_2: wait_for_clean 2023-12-01T06:31:14.160 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1642: wait_for_clean: local cmd= 2023-12-01T06:31:14.160 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1643: wait_for_clean: local num_active_clean=-1 2023-12-01T06:31:14.160 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1644: wait_for_clean: local cur_active_clean 2023-12-01T06:31:14.160 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1645: wait_for_clean: get_timeout_delays 90 .1 2023-12-01T06:31:14.160 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1588: get_timeout_delays: shopt -q -o xtrace 2023-12-01T06:31:14.160 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1588: get_timeout_delays: echo true 2023-12-01T06:31:14.161 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1588: get_timeout_delays: local trace=true 2023-12-01T06:31:14.161 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1589: get_timeout_delays: true 2023-12-01T06:31:14.161 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1589: get_timeout_delays: shopt -u -o xtrace 2023-12-01T06:31:14.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1645: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2023-12-01T06:31:14.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1645: wait_for_clean: local -a delays 2023-12-01T06:31:14.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1646: wait_for_clean: local -i loop=0 2023-12-01T06:31:14.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: flush_pg_stats 2023-12-01T06:31:14.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2197: flush_pg_stats: local timeout=300 2023-12-01T06:31:14.281 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2199: flush_pg_stats: ceph osd ls 2023-12-01T06:31:14.794 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2199: flush_pg_stats: ids='0 2023-12-01T06:31:14.794 INFO:tasks.workunit.client.0.smithi170.stderr:1 2023-12-01T06:31:14.795 INFO:tasks.workunit.client.0.smithi170.stderr:2 2023-12-01T06:31:14.795 INFO:tasks.workunit.client.0.smithi170.stderr:3 2023-12-01T06:31:14.795 INFO:tasks.workunit.client.0.smithi170.stderr:4 2023-12-01T06:31:14.795 INFO:tasks.workunit.client.0.smithi170.stderr:5 2023-12-01T06:31:14.795 INFO:tasks.workunit.client.0.smithi170.stderr:6 2023-12-01T06:31:14.795 INFO:tasks.workunit.client.0.smithi170.stderr:7' 2023-12-01T06:31:14.795 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2200: flush_pg_stats: seqs= 2023-12-01T06:31:14.795 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:31:14.795 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2023-12-01T06:31:15.040 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=21474836499 2023-12-01T06:31:15.040 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 21474836499 2023-12-01T06:31:15.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836499' 2023-12-01T06:31:15.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:31:15.041 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2023-12-01T06:31:15.310 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=42949672977 2023-12-01T06:31:15.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 42949672977 2023-12-01T06:31:15.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836499 1-42949672977' 2023-12-01T06:31:15.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:31:15.312 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2023-12-01T06:31:15.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=64424509454 2023-12-01T06:31:15.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 64424509454 2023-12-01T06:31:15.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836499 1-42949672977 2-64424509454' 2023-12-01T06:31:15.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:31:15.459 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2023-12-01T06:31:15.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=85899345932 2023-12-01T06:31:15.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 85899345932 2023-12-01T06:31:15.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836499 1-42949672977 2-64424509454 3-85899345932' 2023-12-01T06:31:15.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:31:15.601 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2023-12-01T06:31:15.752 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=107374182410 2023-12-01T06:31:15.753 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 107374182410 2023-12-01T06:31:15.753 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836499 1-42949672977 2-64424509454 3-85899345932 4-107374182410' 2023-12-01T06:31:15.753 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:31:15.753 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2023-12-01T06:31:15.922 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=128849018887 2023-12-01T06:31:15.922 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 128849018887 2023-12-01T06:31:15.922 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836499 1-42949672977 2-64424509454 3-85899345932 4-107374182410 5-128849018887' 2023-12-01T06:31:15.922 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:31:15.922 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2023-12-01T06:31:16.081 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=150323855365 2023-12-01T06:31:16.081 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 150323855365 2023-12-01T06:31:16.081 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836499 1-42949672977 2-64424509454 3-85899345932 4-107374182410 5-128849018887 6-150323855365' 2023-12-01T06:31:16.081 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:31:16.081 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2023-12-01T06:31:16.222 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=171798691842 2023-12-01T06:31:16.223 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 171798691842 2023-12-01T06:31:16.223 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836499 1-42949672977 2-64424509454 3-85899345932 4-107374182410 5-128849018887 6-150323855365 7-171798691842' 2023-12-01T06:31:16.223 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:31:16.223 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 0-21474836499 2023-12-01T06:31:16.223 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:31:16.223 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=0 2023-12-01T06:31:16.224 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 0-21474836499 2023-12-01T06:31:16.224 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:31:16.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=21474836499 2023-12-01T06:31:16.225 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.0 seq 21474836499 2023-12-01T06:31:16.226 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2023-12-01T06:31:16.226 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 0 2023-12-01T06:31:16.526 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 21474836498 -lt 21474836499 2023-12-01T06:31:16.526 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2215: flush_pg_stats: sleep 1 2023-12-01T06:31:17.529 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2216: flush_pg_stats: '[' 300 -eq 0 ']' 2023-12-01T06:31:17.530 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 0 2023-12-01T06:31:17.830 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 21474836499 -lt 21474836499 2023-12-01T06:31:17.831 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:31:17.831 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 1-42949672977 2023-12-01T06:31:17.831 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:31:17.833 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=1 2023-12-01T06:31:17.833 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 1-42949672977 2023-12-01T06:31:17.834 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:31:17.835 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=42949672977 2023-12-01T06:31:17.835 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.1 seq 42949672977' 2023-12-01T06:31:17.835 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.1 seq 42949672977 2023-12-01T06:31:17.836 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 1 2023-12-01T06:31:18.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 42949672977 -lt 42949672977 2023-12-01T06:31:18.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:31:18.153 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 2-64424509454 2023-12-01T06:31:18.153 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:31:18.155 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=2 2023-12-01T06:31:18.155 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 2-64424509454 2023-12-01T06:31:18.155 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:31:18.156 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=64424509454 2023-12-01T06:31:18.157 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.2 seq 64424509454 2023-12-01T06:31:18.157 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.2 seq 64424509454' 2023-12-01T06:31:18.157 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 2 2023-12-01T06:31:18.473 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 64424509454 -lt 64424509454 2023-12-01T06:31:18.473 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:31:18.474 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 3-85899345932 2023-12-01T06:31:18.474 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:31:18.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=3 2023-12-01T06:31:18.477 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 3-85899345932 2023-12-01T06:31:18.477 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:31:18.478 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=85899345932 2023-12-01T06:31:18.478 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.3 seq 85899345932 2023-12-01T06:31:18.478 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.3 seq 85899345932' 2023-12-01T06:31:18.479 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 3 2023-12-01T06:31:18.795 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 85899345932 -lt 85899345932 2023-12-01T06:31:18.796 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:31:18.796 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 4-107374182410 2023-12-01T06:31:18.796 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:31:18.798 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=4 2023-12-01T06:31:18.799 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 4-107374182410 2023-12-01T06:31:18.799 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:31:18.800 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=107374182410 2023-12-01T06:31:18.800 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.4 seq 107374182410 2023-12-01T06:31:18.800 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.4 seq 107374182410' 2023-12-01T06:31:18.800 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 4 2023-12-01T06:31:19.108 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 107374182410 -lt 107374182410 2023-12-01T06:31:19.109 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:31:19.109 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 5-128849018887 2023-12-01T06:31:19.110 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:31:19.114 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=5 2023-12-01T06:31:19.114 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 5-128849018887 2023-12-01T06:31:19.114 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:31:19.114 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=128849018887 2023-12-01T06:31:19.114 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.5 seq 128849018887' 2023-12-01T06:31:19.114 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.5 seq 128849018887 2023-12-01T06:31:19.115 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 5 2023-12-01T06:31:19.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 128849018888 -lt 128849018887 2023-12-01T06:31:19.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:31:19.433 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 6-150323855365 2023-12-01T06:31:19.433 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:31:19.434 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=6 2023-12-01T06:31:19.434 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 6-150323855365 2023-12-01T06:31:19.435 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:31:19.435 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=150323855365 2023-12-01T06:31:19.436 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.6 seq 150323855365 2023-12-01T06:31:19.436 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.6 seq 150323855365' 2023-12-01T06:31:19.436 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 6 2023-12-01T06:31:19.780 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 150323855365 -lt 150323855365 2023-12-01T06:31:19.780 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:31:19.781 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:31:19.781 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 7-171798691842 2023-12-01T06:31:19.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=7 2023-12-01T06:31:19.782 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 7-171798691842 2023-12-01T06:31:19.782 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:31:19.784 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=171798691842 2023-12-01T06:31:19.784 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.7 seq 171798691842' 2023-12-01T06:31:19.784 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.7 seq 171798691842 2023-12-01T06:31:19.785 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 7 2023-12-01T06:31:20.101 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 171798691843 -lt 171798691842 2023-12-01T06:31:20.102 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: get_num_pgs 2023-12-01T06:31:20.103 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2023-12-01T06:31:20.103 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2023-12-01T06:31:20.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: test 32 == 0 2023-12-01T06:31:20.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: true 2023-12-01T06:31:20.460 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: get_num_active_clean 2023-12-01T06:31:20.460 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2023-12-01T06:31:20.460 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2023-12-01T06:31:20.460 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2023-12-01T06:31:20.461 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2023-12-01T06:31:20.461 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2023-12-01T06:31:20.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: cur_active_clean=32 2023-12-01T06:31:20.811 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: get_num_pgs 2023-12-01T06:31:20.811 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2023-12-01T06:31:20.811 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2023-12-01T06:31:21.156 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: test 32 = 32 2023-12-01T06:31:21.156 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: break 2023-12-01T06:31:21.156 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: return 0 2023-12-01T06:31:21.156 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:258: TEST_recovery_scrub_2: ceph osd dump 2023-12-01T06:31:21.156 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:258: TEST_recovery_scrub_2: awk '{ print $2 }' 2023-12-01T06:31:21.157 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:258: TEST_recovery_scrub_2: grep '^pool.*['\'']test['\'']' 2023-12-01T06:31:21.461 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:258: TEST_recovery_scrub_2: poolid=1 2023-12-01T06:31:21.461 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:260: TEST_recovery_scrub_2: dd if=/dev/urandom of=testdata.47198 bs=1M count=50 2023-12-01T06:31:21.631 INFO:tasks.workunit.client.0.smithi170.stderr:50+0 records in 2023-12-01T06:31:21.631 INFO:tasks.workunit.client.0.smithi170.stderr:50+0 records out 2023-12-01T06:31:21.631 INFO:tasks.workunit.client.0.smithi170.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.169841 s, 309 MB/s 2023-12-01T06:31:21.632 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: seq 1 40 2023-12-01T06:31:21.633 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:21.633 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj1 testdata.47198 2023-12-01T06:31:23.193 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:23.193 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj2 testdata.47198 2023-12-01T06:31:24.791 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:24.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj3 testdata.47198 2023-12-01T06:31:26.373 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:26.373 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj4 testdata.47198 2023-12-01T06:31:28.148 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:28.149 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj5 testdata.47198 2023-12-01T06:31:29.739 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:29.739 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj6 testdata.47198 2023-12-01T06:31:31.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:31.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj7 testdata.47198 2023-12-01T06:31:32.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:32.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj8 testdata.47198 2023-12-01T06:31:34.434 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:34.435 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj9 testdata.47198 2023-12-01T06:31:36.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:36.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj10 testdata.47198 2023-12-01T06:31:37.775 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:37.775 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj11 testdata.47198 2023-12-01T06:31:39.391 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:39.391 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj12 testdata.47198 2023-12-01T06:31:40.949 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:40.949 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj13 testdata.47198 2023-12-01T06:31:42.489 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:42.489 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj14 testdata.47198 2023-12-01T06:31:44.072 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:44.073 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj15 testdata.47198 2023-12-01T06:31:45.696 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:45.697 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj16 testdata.47198 2023-12-01T06:31:47.301 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:47.301 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj17 testdata.47198 2023-12-01T06:31:48.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:48.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj18 testdata.47198 2023-12-01T06:31:50.376 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:50.376 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj19 testdata.47198 2023-12-01T06:31:52.034 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:52.034 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj20 testdata.47198 2023-12-01T06:31:54.034 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:54.034 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj21 testdata.47198 2023-12-01T06:31:55.656 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:55.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj22 testdata.47198 2023-12-01T06:31:57.261 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:57.261 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj23 testdata.47198 2023-12-01T06:31:58.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:31:58.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj24 testdata.47198 2023-12-01T06:32:00.497 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:00.497 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj25 testdata.47198 2023-12-01T06:32:02.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:02.180 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj26 testdata.47198 2023-12-01T06:32:03.803 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:03.804 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj27 testdata.47198 2023-12-01T06:32:05.545 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:05.545 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj28 testdata.47198 2023-12-01T06:32:07.126 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:07.127 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj29 testdata.47198 2023-12-01T06:32:08.669 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:08.670 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj30 testdata.47198 2023-12-01T06:32:10.251 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:10.251 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj31 testdata.47198 2023-12-01T06:32:11.911 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:11.912 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj32 testdata.47198 2023-12-01T06:32:13.569 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:13.570 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj33 testdata.47198 2023-12-01T06:32:15.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:15.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj34 testdata.47198 2023-12-01T06:32:16.776 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:16.776 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj35 testdata.47198 2023-12-01T06:32:18.410 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:18.410 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj36 testdata.47198 2023-12-01T06:32:20.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:20.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj37 testdata.47198 2023-12-01T06:32:21.689 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:21.689 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj38 testdata.47198 2023-12-01T06:32:23.197 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:23.197 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj39 testdata.47198 2023-12-01T06:32:24.723 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:261: TEST_recovery_scrub_2: for i in $(seq 1 $OBJECTS) 2023-12-01T06:32:24.723 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:263: TEST_recovery_scrub_2: rados -p test put obj40 testdata.47198 2023-12-01T06:32:26.328 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:265: TEST_recovery_scrub_2: rm -f testdata.47198 2023-12-01T06:32:26.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:267: TEST_recovery_scrub_2: ceph osd pool set test size 3 2023-12-01T06:32:26.749 INFO:tasks.workunit.client.0.smithi170.stderr:set pool 1 size to 3 2023-12-01T06:32:26.771 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:269: TEST_recovery_scrub_2: ceph pg dump pgs 2023-12-01T06:32:27.070 INFO:tasks.workunit.client.0.smithi170.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2023-12-01T06:32:27.070 INFO:tasks.workunit.client.0.smithi170.stdout:1.1f 2 0 0 0 0 104857600 0 0 26 0 26 active+clean 2023-12-01T06:31:14.885497+0000 47'26 47:52 [5] 5 [5] 5 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.070 INFO:tasks.workunit.client.0.smithi170.stdout:1.1e 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.748904+0000 47'13 47:40 [3] 3 [3] 3 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.070 INFO:tasks.workunit.client.0.smithi170.stdout:1.1d 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.922224+0000 47'13 47:28 [1] 1 [1] 1 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.070 INFO:tasks.workunit.client.0.smithi170.stdout:1.b 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.814819+0000 47'13 47:30 [1] 1 [1] 1 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.070 INFO:tasks.workunit.client.0.smithi170.stdout:1.a 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.922099+0000 47'13 47:30 [1] 1 [1] 1 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.070 INFO:tasks.workunit.client.0.smithi170.stdout:1.9 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:31:14.814317+0000 0'0 47:17 [1] 1 [1] 1 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.070 INFO:tasks.workunit.client.0.smithi170.stdout:1.8 2 0 0 0 0 104857600 0 0 26 0 26 active+clean 2023-12-01T06:31:14.748135+0000 47'26 47:43 [5] 5 [5] 5 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.070 INFO:tasks.workunit.client.0.smithi170.stdout:1.7 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.922720+0000 47'13 47:30 [6] 6 [6] 6 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.6 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.921882+0000 47'13 47:32 [6] 6 [6] 6 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.5 3 0 0 0 0 157286400 0 0 39 0 39 active+clean 2023-12-01T06:31:14.883706+0000 47'39 47:59 [4] 4 [4] 4 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.4 4 0 0 0 0 209715200 0 0 52 0 52 active+clean 2023-12-01T06:31:14.748435+0000 47'52 47:72 [3] 3 [3] 3 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.3 4 0 0 0 0 209715200 0 0 52 0 52 active+clean 2023-12-01T06:31:14.779854+0000 47'52 47:72 [4] 4 [4] 4 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.2 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.157486+0000 47'13 47:33 [0] 0 [0] 0 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.c 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:31:14.748807+0000 0'0 47:17 [3] 3 [3] 3 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.1 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.884835+0000 47'13 47:39 [5] 5 [5] 5 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.1c 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.292887+0000 47'13 47:28 [0] 0 [0] 0 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.0 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.157533+0000 47'13 47:37 [7] 7 [7] 7 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.d 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.291939+0000 47'13 47:27 [7] 7 [7] 7 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.e 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.921889+0000 47'13 47:29 [1] 1 [1] 1 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.f 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.779728+0000 47'13 47:28 [6] 6 [6] 6 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.10 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.885172+0000 47'13 47:29 [5] 5 [5] 5 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.11 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.292357+0000 47'13 47:29 [0] 0 [0] 0 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.071 INFO:tasks.workunit.client.0.smithi170.stdout:1.12 2 0 0 0 0 69206016 0 0 17 0 17 active+clean 2023-12-01T06:31:14.291956+0000 47'17 47:32 [0] 0 [0] 0 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.072 INFO:tasks.workunit.client.0.smithi170.stdout:1.13 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.883611+0000 47'13 47:28 [4] 4 [4] 4 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.072 INFO:tasks.workunit.client.0.smithi170.stdout:1.14 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:31:14.779675+0000 0'0 47:15 [2] 2 [2] 2 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.072 INFO:tasks.workunit.client.0.smithi170.stdout:1.15 3 0 0 0 0 157286400 0 0 39 0 39 active+clean 2023-12-01T06:31:14.815384+0000 47'39 47:54 [1] 1 [1] 1 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.072 INFO:tasks.workunit.client.0.smithi170.stdout:1.16 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:31:14.922845+0000 0'0 47:14 [6] 6 [6] 6 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.072 INFO:tasks.workunit.client.0.smithi170.stdout:1.17 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.292692+0000 47'13 47:28 [0] 0 [0] 0 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.072 INFO:tasks.workunit.client.0.smithi170.stdout:1.18 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:31:14.748239+0000 0'0 47:15 [5] 5 [5] 5 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.072 INFO:tasks.workunit.client.0.smithi170.stdout:1.19 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.748263+0000 47'13 47:28 [3] 3 [3] 3 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.072 INFO:tasks.workunit.client.0.smithi170.stdout:1.1a 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:31:14.815600+0000 0'0 47:15 [1] 1 [1] 1 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.072 INFO:tasks.workunit.client.0.smithi170.stdout:1.1b 1 0 0 0 0 52428800 0 0 13 0 13 active+clean 2023-12-01T06:31:14.292546+0000 47'13 47:28 [0] 0 [0] 0 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:27.072 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:32:27.072 INFO:tasks.workunit.client.0.smithi170.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2023-12-01T06:32:27.072 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:32:27.086 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:275: TEST_recovery_scrub_2: ceph tell osd.1 config get osd_recovery_max_active 2023-12-01T06:32:27.315 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:32:27.316 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_recovery_max_active": "0" 2023-12-01T06:32:27.316 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:32:27.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:277: TEST_recovery_scrub_2: ceph tell osd.1 config get osd_recovery_max_active_hdd 2023-12-01T06:32:27.524 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:32:27.525 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_recovery_max_active_hdd": "3" 2023-12-01T06:32:27.525 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:32:27.541 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:278: TEST_recovery_scrub_2: ceph tell osd.1 config get osd_recovery_max_active_ssd 2023-12-01T06:32:27.731 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:32:27.731 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_recovery_max_active_ssd": "10" 2023-12-01T06:32:27.731 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:32:27.745 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:281: TEST_recovery_scrub_2: count=0 2023-12-01T06:32:27.746 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:282: TEST_recovery_scrub_2: true 2023-12-01T06:32:27.747 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:286: TEST_recovery_scrub_2: ceph --format json pg dump pgs 2023-12-01T06:32:27.747 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:287: TEST_recovery_scrub_2: jq '.pg_stats | [.[].state]' 2023-12-01T06:32:27.748 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:287: TEST_recovery_scrub_2: grep recovering 2023-12-01T06:32:27.748 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:287: TEST_recovery_scrub_2: wc -l 2023-12-01T06:32:28.086 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:32:28.104 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:286: TEST_recovery_scrub_2: test 0 -ge 2 2023-12-01T06:32:28.104 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:290: TEST_recovery_scrub_2: sleep 2 2023-12-01T06:32:30.106 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:291: TEST_recovery_scrub_2: test 0 -eq 10 2023-12-01T06:32:30.106 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:296: TEST_recovery_scrub_2: expr 0 + 1 2023-12-01T06:32:30.108 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:296: TEST_recovery_scrub_2: count=1 2023-12-01T06:32:30.109 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:282: TEST_recovery_scrub_2: true 2023-12-01T06:32:30.111 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:286: TEST_recovery_scrub_2: ceph --format json pg dump pgs 2023-12-01T06:32:30.111 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:287: TEST_recovery_scrub_2: jq '.pg_stats | [.[].state]' 2023-12-01T06:32:30.111 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:287: TEST_recovery_scrub_2: grep recovering 2023-12-01T06:32:30.111 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:287: TEST_recovery_scrub_2: wc -l 2023-12-01T06:32:30.410 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:32:30.428 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:286: TEST_recovery_scrub_2: test 0 -ge 2 2023-12-01T06:32:30.429 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:290: TEST_recovery_scrub_2: sleep 2 2023-12-01T06:32:32.430 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:291: TEST_recovery_scrub_2: test 1 -eq 10 2023-12-01T06:32:32.431 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:296: TEST_recovery_scrub_2: expr 1 + 1 2023-12-01T06:32:32.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:296: TEST_recovery_scrub_2: count=2 2023-12-01T06:32:32.433 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:282: TEST_recovery_scrub_2: true 2023-12-01T06:32:32.435 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:286: TEST_recovery_scrub_2: ceph --format json pg dump pgs 2023-12-01T06:32:32.435 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:287: TEST_recovery_scrub_2: jq '.pg_stats | [.[].state]' 2023-12-01T06:32:32.436 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:287: TEST_recovery_scrub_2: grep recovering 2023-12-01T06:32:32.436 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:287: TEST_recovery_scrub_2: wc -l 2023-12-01T06:32:32.735 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:32:32.754 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:286: TEST_recovery_scrub_2: test 1 -ge 2 2023-12-01T06:32:32.754 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:290: TEST_recovery_scrub_2: sleep 2 2023-12-01T06:32:34.755 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:291: TEST_recovery_scrub_2: test 2 -eq 10 2023-12-01T06:32:34.756 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:296: TEST_recovery_scrub_2: expr 2 + 1 2023-12-01T06:32:34.757 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:296: TEST_recovery_scrub_2: count=3 2023-12-01T06:32:34.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:282: TEST_recovery_scrub_2: true 2023-12-01T06:32:34.760 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:287: TEST_recovery_scrub_2: jq '.pg_stats | [.[].state]' 2023-12-01T06:32:34.760 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:286: TEST_recovery_scrub_2: ceph --format json pg dump pgs 2023-12-01T06:32:34.760 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:287: TEST_recovery_scrub_2: grep recovering 2023-12-01T06:32:34.761 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:287: TEST_recovery_scrub_2: wc -l 2023-12-01T06:32:35.058 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:32:35.076 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:286: TEST_recovery_scrub_2: test 2 -ge 2 2023-12-01T06:32:35.076 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:288: TEST_recovery_scrub_2: break 2023-12-01T06:32:35.076 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:298: TEST_recovery_scrub_2: ceph pg dump pgs 2023-12-01T06:32:35.360 INFO:tasks.workunit.client.0.smithi170.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2023-12-01T06:32:35.360 INFO:tasks.workunit.client.0.smithi170.stdout:1.1f 2 0 4 0 0 104857600 0 0 26 0 26 active+recovery_wait+degraded 2023-12-01T06:32:28.839130+0000 47'26 50:62 [5,0,2] 5 [5,0,2] 5 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.361 INFO:tasks.workunit.client.0.smithi170.stdout:1.1e 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:29.160190+0000 47'13 50:50 [3,0,7] 3 [3,0,7] 3 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.361 INFO:tasks.workunit.client.0.smithi170.stdout:1.1d 2 0 4 0 0 104857600 0 0 26 0 26 active+recovery_wait+degraded 2023-12-01T06:32:28.538879+0000 47'26 50:49 [1,3,0] 1 [1,3,0] 1 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.361 INFO:tasks.workunit.client.0.smithi170.stdout:1.b 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:28.636329+0000 47'13 50:39 [1,0,2] 1 [1,0,2] 1 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.361 INFO:tasks.workunit.client.0.smithi170.stdout:1.a 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:28.634279+0000 47'13 50:39 [1,6,2] 1 [1,6,2] 1 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.361 INFO:tasks.workunit.client.0.smithi170.stdout:1.9 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:32:28.357421+0000 0'0 50:28 [1,4,5] 1 [1,4,5] 1 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.361 INFO:tasks.workunit.client.0.smithi170.stdout:1.8 2 0 4 0 0 104857600 0 0 26 0 26 active+recovery_wait+degraded 2023-12-01T06:32:28.599771+0000 47'26 50:52 [5,4,3] 5 [5,4,3] 5 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.361 INFO:tasks.workunit.client.0.smithi170.stdout:1.7 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:29.160150+0000 47'13 50:39 [6,7,3] 6 [6,7,3] 6 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.361 INFO:tasks.workunit.client.0.smithi170.stdout:1.6 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:29.160419+0000 47'13 50:40 [6,7,3] 6 [6,7,3] 6 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.361 INFO:tasks.workunit.client.0.smithi170.stdout:1.5 3 0 6 0 0 157286400 0 0 39 0 39 active+recovery_wait+degraded 2023-12-01T06:32:28.837587+0000 47'39 50:67 [4,2,6] 4 [4,2,6] 4 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.361 INFO:tasks.workunit.client.0.smithi170.stdout:1.4 4 0 8 0 0 209715200 0 0 52 0 52 active+recovering+degraded 2023-12-01T06:32:28.784362+0000 47'52 50:82 [3,0,6] 3 [3,0,6] 3 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.361 INFO:tasks.workunit.client.0.smithi170.stdout:1.3 4 0 8 0 0 209715200 0 0 52 0 52 active+recovery_wait+degraded 2023-12-01T06:32:28.635105+0000 47'52 50:80 [4,1,2] 4 [4,1,2] 4 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.361 INFO:tasks.workunit.client.0.smithi170.stdout:1.2 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:29.329296+0000 47'13 50:43 [0,7,1] 0 [0,7,1] 0 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.361 INFO:tasks.workunit.client.0.smithi170.stdout:1.c 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:32:28.418368+0000 0'0 50:27 [3,0,4] 3 [3,0,4] 3 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.361 INFO:tasks.workunit.client.0.smithi170.stdout:1.1 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:28.839537+0000 47'13 50:49 [5,0,4] 5 [5,0,4] 5 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.362 INFO:tasks.workunit.client.0.smithi170.stdout:1.1c 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:28.782262+0000 47'13 50:37 [0,3,4] 0 [0,3,4] 0 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.362 INFO:tasks.workunit.client.0.smithi170.stdout:1.0 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:28.781104+0000 47'13 50:48 [7,0,2] 7 [7,0,2] 7 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.362 INFO:tasks.workunit.client.0.smithi170.stdout:1.d 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:28.599598+0000 47'13 50:37 [7,4,2] 7 [7,4,2] 7 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.362 INFO:tasks.workunit.client.0.smithi170.stdout:1.e 1 0 2 0 0 52428800 0 0 13 0 13 active+recovering+degraded 2023-12-01T06:32:28.420972+0000 47'13 50:39 [1,4,2] 1 [1,4,2] 1 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.362 INFO:tasks.workunit.client.0.smithi170.stdout:1.f 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:29.020702+0000 47'13 50:36 [6,0,3] 6 [6,0,3] 6 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.362 INFO:tasks.workunit.client.0.smithi170.stdout:1.10 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:29.329413+0000 47'13 50:37 [5,7,6] 5 [5,7,6] 5 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.362 INFO:tasks.workunit.client.0.smithi170.stdout:1.11 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:28.715449+0000 47'13 50:38 [0,1,3] 0 [0,1,3] 0 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.362 INFO:tasks.workunit.client.0.smithi170.stdout:1.12 2 0 4 0 0 104857600 0 0 26 0 26 active+recovery_wait+degraded 2023-12-01T06:32:28.715191+0000 47'26 50:51 [0,1,6] 0 [0,1,6] 0 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.362 INFO:tasks.workunit.client.0.smithi170.stdout:1.13 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:28.715792+0000 47'13 50:37 [4,6,3] 4 [4,6,3] 4 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.362 INFO:tasks.workunit.client.0.smithi170.stdout:1.14 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:32:29.020697+0000 0'0 50:26 [2,0,4] 2 [2,0,4] 2 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.362 INFO:tasks.workunit.client.0.smithi170.stdout:1.15 3 0 6 0 0 157286400 0 0 39 0 39 active+recovery_wait+degraded 2023-12-01T06:32:28.540022+0000 47'39 50:63 [1,5,3] 1 [1,5,3] 1 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.362 INFO:tasks.workunit.client.0.smithi170.stdout:1.16 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:32:28.968980+0000 0'0 50:25 [6,0,7] 6 [6,0,7] 6 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.362 INFO:tasks.workunit.client.0.smithi170.stdout:1.17 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:28.966688+0000 47'13 50:37 [0,4,5] 0 [0,4,5] 0 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.362 INFO:tasks.workunit.client.0.smithi170.stdout:1.18 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:32:28.538983+0000 0'0 50:25 [5,2,3] 5 [5,2,3] 5 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.363 INFO:tasks.workunit.client.0.smithi170.stdout:1.19 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:28.838137+0000 47'13 50:36 [3,0,4] 3 [3,0,4] 3 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.363 INFO:tasks.workunit.client.0.smithi170.stdout:1.1a 0 0 0 0 0 0 0 0 0 0 0 active+clean 2023-12-01T06:32:28.968125+0000 0'0 50:25 [1,7,4] 1 [1,7,4] 1 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.363 INFO:tasks.workunit.client.0.smithi170.stdout:1.1b 1 0 2 0 0 52428800 0 0 13 0 13 active+recovery_wait+degraded 2023-12-01T06:32:28.634146+0000 47'13 50:37 [0,1,2] 0 [0,1,2] 0 0'0 2023-12-01T06:31:13.087675+0000 0'0 2023-12-01T06:31:13.087675+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:31:13.087675+0000 0 0 2023-12-01T06:32:35.363 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:32:35.363 INFO:tasks.workunit.client.0.smithi170.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2023-12-01T06:32:35.363 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:32:35.375 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:300: TEST_recovery_scrub_2: pids= 2023-12-01T06:32:35.375 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:301: TEST_recovery_scrub_2: recov_scrub_count=0 2023-12-01T06:32:35.376 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: expr 32 - 1 2023-12-01T06:32:35.378 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: seq 0 31 2023-12-01T06:32:35.378 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.379 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 0 2023-12-01T06:32:35.380 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.0 2023-12-01T06:32:35.380 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.380 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.380 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68759"' 2023-12-01T06:32:35.380 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68759' 2023-12-01T06:32:35.380 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.0 2023-12-01T06:32:35.382 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 1 2023-12-01T06:32:35.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68762: /' 2023-12-01T06:32:35.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.1 2023-12-01T06:32:35.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68764"' 2023-12-01T06:32:35.382 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68764' 2023-12-01T06:32:35.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.1 2023-12-01T06:32:35.382 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 2 2023-12-01T06:32:35.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68769: /' 2023-12-01T06:32:35.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.2 2023-12-01T06:32:35.383 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.383 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68771"' 2023-12-01T06:32:35.384 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68771' 2023-12-01T06:32:35.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.2 2023-12-01T06:32:35.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68776: /' 2023-12-01T06:32:35.384 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 3 2023-12-01T06:32:35.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.3 2023-12-01T06:32:35.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68779"' 2023-12-01T06:32:35.384 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68779' 2023-12-01T06:32:35.385 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.385 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 4 2023-12-01T06:32:35.385 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.3 2023-12-01T06:32:35.385 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.4 2023-12-01T06:32:35.385 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.385 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.385 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68784"' 2023-12-01T06:32:35.385 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68784' 2023-12-01T06:32:35.386 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.388 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68783: /' 2023-12-01T06:32:35.388 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 5 2023-12-01T06:32:35.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.5 2023-12-01T06:32:35.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68787"' 2023-12-01T06:32:35.389 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68787' 2023-12-01T06:32:35.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.390 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68791: /' 2023-12-01T06:32:35.390 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.5 2023-12-01T06:32:35.391 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68792: /' 2023-12-01T06:32:35.400 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 6 2023-12-01T06:32:35.401 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.6 2023-12-01T06:32:35.401 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.401 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.401 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68798"' 2023-12-01T06:32:35.401 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68798' 2023-12-01T06:32:35.401 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.4 2023-12-01T06:32:35.408 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 7 2023-12-01T06:32:35.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.7 2023-12-01T06:32:35.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68803"' 2023-12-01T06:32:35.409 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68803' 2023-12-01T06:32:35.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.418 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68801: /' 2023-12-01T06:32:35.420 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.6 2023-12-01T06:32:35.424 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.7 2023-12-01T06:32:35.426 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68806: /' 2023-12-01T06:32:35.428 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 8 2023-12-01T06:32:35.429 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.8 2023-12-01T06:32:35.429 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.429 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.429 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68815"' 2023-12-01T06:32:35.429 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68815' 2023-12-01T06:32:35.429 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.436 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 9 2023-12-01T06:32:35.437 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.9 2023-12-01T06:32:35.437 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.437 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.438 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68817"' 2023-12-01T06:32:35.438 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68817' 2023-12-01T06:32:35.438 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.452 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68820: /' 2023-12-01T06:32:35.456 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 10 2023-12-01T06:32:35.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.9 2023-12-01T06:32:35.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.a 2023-12-01T06:32:35.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.457 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68824"' 2023-12-01T06:32:35.457 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68824' 2023-12-01T06:32:35.457 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.8 2023-12-01T06:32:35.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68822: /' 2023-12-01T06:32:35.463 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 11 2023-12-01T06:32:35.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.b 2023-12-01T06:32:35.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68827"' 2023-12-01T06:32:35.464 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68827' 2023-12-01T06:32:35.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.468 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.a 2023-12-01T06:32:35.480 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.b 2023-12-01T06:32:35.484 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68830: /' 2023-12-01T06:32:35.484 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 12 2023-12-01T06:32:35.485 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.c 2023-12-01T06:32:35.485 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.485 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.485 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68839"' 2023-12-01T06:32:35.485 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68839' 2023-12-01T06:32:35.485 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.491 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68837: /' 2023-12-01T06:32:35.500 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 13 2023-12-01T06:32:35.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.d 2023-12-01T06:32:35.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68847"' 2023-12-01T06:32:35.501 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68847' 2023-12-01T06:32:35.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.508 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68846: /' 2023-12-01T06:32:35.508 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 14 2023-12-01T06:32:35.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.e 2023-12-01T06:32:35.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68849"' 2023-12-01T06:32:35.509 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68849' 2023-12-01T06:32:35.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.516 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.c 2023-12-01T06:32:35.516 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 15 2023-12-01T06:32:35.517 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.f 2023-12-01T06:32:35.517 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.517 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.517 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68852"' 2023-12-01T06:32:35.517 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68852' 2023-12-01T06:32:35.517 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.524 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 16 2023-12-01T06:32:35.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.10 2023-12-01T06:32:35.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68856"' 2023-12-01T06:32:35.525 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68856' 2023-12-01T06:32:35.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.528 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68855: /' 2023-12-01T06:32:35.528 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.e 2023-12-01T06:32:35.529 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.d 2023-12-01T06:32:35.535 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 17 2023-12-01T06:32:35.536 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.11 2023-12-01T06:32:35.536 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.537 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.537 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68864"' 2023-12-01T06:32:35.537 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68864' 2023-12-01T06:32:35.537 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.537 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68859: /' 2023-12-01T06:32:35.540 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.f 2023-12-01T06:32:35.548 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 18 2023-12-01T06:32:35.548 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68869: /' 2023-12-01T06:32:35.550 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.11 2023-12-01T06:32:35.556 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68861: /' 2023-12-01T06:32:35.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.12 2023-12-01T06:32:35.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68874: /' 2023-12-01T06:32:35.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68880"' 2023-12-01T06:32:35.557 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68880' 2023-12-01T06:32:35.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.10 2023-12-01T06:32:35.558 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 19 2023-12-01T06:32:35.558 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.13 2023-12-01T06:32:35.558 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.558 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.559 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68883"' 2023-12-01T06:32:35.559 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68883' 2023-12-01T06:32:35.559 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.559 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.12 2023-12-01T06:32:35.576 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 20 2023-12-01T06:32:35.578 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.14 2023-12-01T06:32:35.578 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.578 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.578 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68893"' 2023-12-01T06:32:35.578 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68893' 2023-12-01T06:32:35.578 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.578 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.13 2023-12-01T06:32:35.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68886: /' 2023-12-01T06:32:35.592 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 21 2023-12-01T06:32:35.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.15 2023-12-01T06:32:35.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68899"' 2023-12-01T06:32:35.593 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68899' 2023-12-01T06:32:35.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68892: /' 2023-12-01T06:32:35.610 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 22 2023-12-01T06:32:35.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.16 2023-12-01T06:32:35.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68905"' 2023-12-01T06:32:35.610 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68905' 2023-12-01T06:32:35.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.14 2023-12-01T06:32:35.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68902: /' 2023-12-01T06:32:35.621 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68908: /' 2023-12-01T06:32:35.625 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.15 2023-12-01T06:32:35.628 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 23 2023-12-01T06:32:35.629 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.17 2023-12-01T06:32:35.629 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.629 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.629 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68916"' 2023-12-01T06:32:35.629 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68916' 2023-12-01T06:32:35.629 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.633 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.16 2023-12-01T06:32:35.636 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 24 2023-12-01T06:32:35.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68911: /' 2023-12-01T06:32:35.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.18 2023-12-01T06:32:35.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68925"' 2023-12-01T06:32:35.653 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68925' 2023-12-01T06:32:35.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.660 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68920: /' 2023-12-01T06:32:35.661 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.17 2023-12-01T06:32:35.672 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 25 2023-12-01T06:32:35.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.19 2023-12-01T06:32:35.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.674 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68930"' 2023-12-01T06:32:35.674 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68930' 2023-12-01T06:32:35.674 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.18 2023-12-01T06:32:35.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68929: /' 2023-12-01T06:32:35.685 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 26 2023-12-01T06:32:35.685 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.1a 2023-12-01T06:32:35.685 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.685 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.685 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68935"' 2023-12-01T06:32:35.685 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68935' 2023-12-01T06:32:35.685 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.700 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68939: /' 2023-12-01T06:32:35.700 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 27 2023-12-01T06:32:35.701 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.19 2023-12-01T06:32:35.702 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.1b 2023-12-01T06:32:35.702 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.702 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.702 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68944"' 2023-12-01T06:32:35.702 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68944' 2023-12-01T06:32:35.702 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.720 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.1a 2023-12-01T06:32:35.720 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 28 2023-12-01T06:32:35.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.1c 2023-12-01T06:32:35.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68954"' 2023-12-01T06:32:35.723 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68954' 2023-12-01T06:32:35.723 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.724 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68943: /' 2023-12-01T06:32:35.740 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.1b 2023-12-01T06:32:35.744 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.1c 2023-12-01T06:32:35.752 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68953: /' 2023-12-01T06:32:35.759 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68960: /' 2023-12-01T06:32:35.760 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 29 2023-12-01T06:32:35.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.1d 2023-12-01T06:32:35.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68966"' 2023-12-01T06:32:35.761 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68966' 2023-12-01T06:32:35.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.776 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 30 2023-12-01T06:32:35.777 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.1e 2023-12-01T06:32:35.777 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.777 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.777 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68975"' 2023-12-01T06:32:35.778 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68975' 2023-12-01T06:32:35.778 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:302: TEST_recovery_scrub_2: for pg in $(seq 0 $(expr $PGS - 1)) 2023-12-01T06:32:35.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.1d 2023-12-01T06:32:35.792 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: printf %x 31 2023-12-01T06:32:35.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:304: TEST_recovery_scrub_2: run_in_background pids pg_scrub_mod 1.1f 2023-12-01T06:32:35.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: run_in_background: local pid_variable=pids 2023-12-01T06:32:35.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2128: run_in_background: shift 2023-12-01T06:32:35.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: eval 'pids+=" 68983"' 2023-12-01T06:32:35.793 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: run_in_background: pids+=' 68983' 2023-12-01T06:32:35.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:306: TEST_recovery_scrub_2: wait_background_check pids 2023-12-01T06:32:35.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:210: wait_background_check: pids=' 68759 68764 68771 68779 68784 68787 68798 68803 68815 68817 68824 68827 68839 68847 68849 68852 68856 68864 68880 68883 68893 68899 68905 68916 68925 68930 68935 68944 68954 68966 68975 68983' 2023-12-01T06:32:35.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:212: wait_background_check: return_code=0 2023-12-01T06:32:35.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:35.794 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68759 2023-12-01T06:32:35.796 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68969: /' 2023-12-01T06:32:35.824 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68985: /' 2023-12-01T06:32:35.825 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.1e 2023-12-01T06:32:35.830 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: sed 's/^/68993: /' 2023-12-01T06:32:35.844 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: pg_scrub_mod 1.1f 2023-12-01T06:32:41.413 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.4 2023-12-01T06:32:41.413 INFO:tasks.workunit.client.0.smithi170.stderr:68791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.4 2023-12-01T06:32:41.413 INFO:tasks.workunit.client.0.smithi170.stderr:68791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.4 2023-12-01T06:32:41.413 INFO:tasks.workunit.client.0.smithi170.stderr:68791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:41.413 INFO:tasks.workunit.client.0.smithi170.stderr:68791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.4") | .last_scrub_stamp' 2023-12-01T06:32:41.413 INFO:tasks.workunit.client.0.smithi170.stderr:68791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:41.413 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:41.413 INFO:tasks.workunit.client.0.smithi170.stderr:68791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.4 query 2023-12-01T06:32:41.413 INFO:tasks.workunit.client.0.smithi170.stderr:68791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:41.413 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:41.413 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:41.413 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.4 2023-12-01T06:32:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:68791: instructing pg 1.4 on osd.3 to scrub 2023-12-01T06:32:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.4") | .state' 2023-12-01T06:32:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:68791: dumped pgs 2023-12-01T06:32:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:198: pg_scrub_mod: recovery=true 2023-12-01T06:32:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.4 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.4 2023-12-01T06:32:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:41.415 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:41.415 INFO:tasks.workunit.client.0.smithi170.stderr:68791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.4 last_scrub_stamp 2023-12-01T06:32:41.415 INFO:tasks.workunit.client.0.smithi170.stderr:68791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.4 2023-12-01T06:32:41.415 INFO:tasks.workunit.client.0.smithi170.stderr:68791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:41.415 INFO:tasks.workunit.client.0.smithi170.stderr:68791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.4") | .last_scrub_stamp' 2023-12-01T06:32:42.300 INFO:tasks.workunit.client.0.smithi170.stderr:68791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.e 2023-12-01T06:32:42.300 INFO:tasks.workunit.client.0.smithi170.stderr:68861: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.e 2023-12-01T06:32:42.300 INFO:tasks.workunit.client.0.smithi170.stderr:68861: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.e 2023-12-01T06:32:42.300 INFO:tasks.workunit.client.0.smithi170.stderr:68861: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.e") | .last_scrub_stamp' 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.e query 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.e 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: instructing pg 1.e on osd.1 to scrub 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.e") | .state' 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: dumped pgs 2023-12-01T06:32:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:198: pg_scrub_mod: recovery=true 2023-12-01T06:32:42.302 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.e 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:42.302 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.e 2023-12-01T06:32:42.302 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:42.302 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:42.302 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:42.302 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:42.302 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:42.302 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:42.302 INFO:tasks.workunit.client.0.smithi170.stderr:68861: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.e last_scrub_stamp 2023-12-01T06:32:42.302 INFO:tasks.workunit.client.0.smithi170.stderr:68861: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.e 2023-12-01T06:32:42.302 INFO:tasks.workunit.client.0.smithi170.stderr:68861: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:42.302 INFO:tasks.workunit.client.0.smithi170.stderr:68861: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.e") | .last_scrub_stamp' 2023-12-01T06:32:43.108 INFO:tasks.workunit.client.0.smithi170.stderr:68861: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.3 2023-12-01T06:32:43.108 INFO:tasks.workunit.client.0.smithi170.stderr:68783: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.3 2023-12-01T06:32:43.108 INFO:tasks.workunit.client.0.smithi170.stderr:68783: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2023-12-01T06:32:43.108 INFO:tasks.workunit.client.0.smithi170.stderr:68783: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:43.108 INFO:tasks.workunit.client.0.smithi170.stderr:68783: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2023-12-01T06:32:43.108 INFO:tasks.workunit.client.0.smithi170.stderr:68783: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:43.108 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.108 INFO:tasks.workunit.client.0.smithi170.stderr:68783: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.3 query 2023-12-01T06:32:43.108 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:43.108 INFO:tasks.workunit.client.0.smithi170.stderr:68783: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:43.109 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:43.109 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:43.109 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.3 2023-12-01T06:32:43.109 INFO:tasks.workunit.client.0.smithi170.stderr:68783: instructing pg 1.3 on osd.4 to scrub 2023-12-01T06:32:43.109 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:43.109 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:43.109 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.3") | .state' 2023-12-01T06:32:43.109 INFO:tasks.workunit.client.0.smithi170.stderr:68783: dumped pgs 2023-12-01T06:32:43.109 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.3 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.109 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.3 2023-12-01T06:32:43.109 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.109 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:43.109 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:43.109 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:43.110 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:43.110 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:43.110 INFO:tasks.workunit.client.0.smithi170.stderr:68783: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.3 last_scrub_stamp 2023-12-01T06:32:43.110 INFO:tasks.workunit.client.0.smithi170.stderr:68783: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2023-12-01T06:32:43.110 INFO:tasks.workunit.client.0.smithi170.stderr:68783: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:43.110 INFO:tasks.workunit.client.0.smithi170.stderr:68783: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2023-12-01T06:32:43.110 INFO:tasks.workunit.client.0.smithi170.stderr:68783: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:43.110 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:43.110 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:43.110 INFO:tasks.workunit.client.0.smithi170.stderr:68783: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:43.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:43.388 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.1 2023-12-01T06:32:43.389 INFO:tasks.workunit.client.0.smithi170.stderr:68769: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.1 2023-12-01T06:32:43.389 INFO:tasks.workunit.client.0.smithi170.stderr:68769: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2023-12-01T06:32:43.389 INFO:tasks.workunit.client.0.smithi170.stderr:68769: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:43.389 INFO:tasks.workunit.client.0.smithi170.stderr:68769: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:43.389 INFO:tasks.workunit.client.0.smithi170.stderr:68769: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2023-12-01T06:32:43.389 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.389 INFO:tasks.workunit.client.0.smithi170.stderr:68769: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:43.389 INFO:tasks.workunit.client.0.smithi170.stderr:68769: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.1 query 2023-12-01T06:32:43.389 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:43.389 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:43.389 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:43.389 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.1 2023-12-01T06:32:43.390 INFO:tasks.workunit.client.0.smithi170.stderr:68769: instructing pg 1.1 on osd.5 to scrub 2023-12-01T06:32:43.390 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.1") | .state' 2023-12-01T06:32:43.390 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:43.390 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:43.390 INFO:tasks.workunit.client.0.smithi170.stderr:68769: dumped pgs 2023-12-01T06:32:43.390 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.1 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.390 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.1 2023-12-01T06:32:43.390 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.390 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:43.390 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:43.390 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:43.390 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:43.390 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:43.390 INFO:tasks.workunit.client.0.smithi170.stderr:68769: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.1 last_scrub_stamp 2023-12-01T06:32:43.391 INFO:tasks.workunit.client.0.smithi170.stderr:68769: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2023-12-01T06:32:43.391 INFO:tasks.workunit.client.0.smithi170.stderr:68769: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:43.391 INFO:tasks.workunit.client.0.smithi170.stderr:68769: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:43.391 INFO:tasks.workunit.client.0.smithi170.stderr:68769: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2023-12-01T06:32:43.391 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:43.391 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:43.391 INFO:tasks.workunit.client.0.smithi170.stderr:68769: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:43.391 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:43.563 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.9 2023-12-01T06:32:43.563 INFO:tasks.workunit.client.0.smithi170.stderr:68822: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.9 2023-12-01T06:32:43.563 INFO:tasks.workunit.client.0.smithi170.stderr:68822: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.9 2023-12-01T06:32:43.563 INFO:tasks.workunit.client.0.smithi170.stderr:68822: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.9") | .last_scrub_stamp' 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.9 query 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.9 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: instructing pg 1.9 on osd.1 to scrub 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.9") | .state' 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: dumped pgs 2023-12-01T06:32:43.564 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.9 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.9 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.9 last_scrub_stamp 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.9 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.9") | .last_scrub_stamp' 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:43.565 INFO:tasks.workunit.client.0.smithi170.stderr:68822: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:43.566 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:43.688 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.2 2023-12-01T06:32:43.688 INFO:tasks.workunit.client.0.smithi170.stderr:68776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.2 2023-12-01T06:32:43.688 INFO:tasks.workunit.client.0.smithi170.stderr:68776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.2 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.2") | .last_scrub_stamp' 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.2 query 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.2 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: instructing pg 1.2 on osd.0 to scrub 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.2") | .state' 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:43.689 INFO:tasks.workunit.client.0.smithi170.stderr:68776: dumped pgs 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.2 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.2 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.2 last_scrub_stamp 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.2 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.2") | .last_scrub_stamp' 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:43.690 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:43.691 INFO:tasks.workunit.client.0.smithi170.stderr:68776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:43.691 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:43.966 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.c 2023-12-01T06:32:43.966 INFO:tasks.workunit.client.0.smithi170.stderr:68846: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.c 2023-12-01T06:32:43.966 INFO:tasks.workunit.client.0.smithi170.stderr:68846: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.c 2023-12-01T06:32:43.966 INFO:tasks.workunit.client.0.smithi170.stderr:68846: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:43.966 INFO:tasks.workunit.client.0.smithi170.stderr:68846: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.c") | .last_scrub_stamp' 2023-12-01T06:32:43.966 INFO:tasks.workunit.client.0.smithi170.stderr:68846: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:43.966 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.966 INFO:tasks.workunit.client.0.smithi170.stderr:68846: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:43.966 INFO:tasks.workunit.client.0.smithi170.stderr:68846: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.c query 2023-12-01T06:32:43.966 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:43.966 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.c 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: instructing pg 1.c on osd.3 to scrub 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.c") | .state' 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: dumped pgs 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.c 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.c 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:43.967 INFO:tasks.workunit.client.0.smithi170.stderr:68846: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.c last_scrub_stamp 2023-12-01T06:32:43.968 INFO:tasks.workunit.client.0.smithi170.stderr:68846: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.c 2023-12-01T06:32:43.968 INFO:tasks.workunit.client.0.smithi170.stderr:68846: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:43.968 INFO:tasks.workunit.client.0.smithi170.stderr:68846: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:43.968 INFO:tasks.workunit.client.0.smithi170.stderr:68846: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.c") | .last_scrub_stamp' 2023-12-01T06:32:43.968 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:43.968 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:43.968 INFO:tasks.workunit.client.0.smithi170.stderr:68846: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:43.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:44.102 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.16 2023-12-01T06:32:44.102 INFO:tasks.workunit.client.0.smithi170.stderr:68911: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.16 2023-12-01T06:32:44.102 INFO:tasks.workunit.client.0.smithi170.stderr:68911: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.16 2023-12-01T06:32:44.102 INFO:tasks.workunit.client.0.smithi170.stderr:68911: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.102 INFO:tasks.workunit.client.0.smithi170.stderr:68911: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.16") | .last_scrub_stamp' 2023-12-01T06:32:44.102 INFO:tasks.workunit.client.0.smithi170.stderr:68911: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.102 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.102 INFO:tasks.workunit.client.0.smithi170.stderr:68911: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:44.102 INFO:tasks.workunit.client.0.smithi170.stderr:68911: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.16 query 2023-12-01T06:32:44.102 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.16 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: instructing pg 1.16 on osd.6 to scrub 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.16") | .state' 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: dumped pgs 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.16 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.16 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:44.103 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:44.104 INFO:tasks.workunit.client.0.smithi170.stderr:68911: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.16 last_scrub_stamp 2023-12-01T06:32:44.104 INFO:tasks.workunit.client.0.smithi170.stderr:68911: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.16 2023-12-01T06:32:44.104 INFO:tasks.workunit.client.0.smithi170.stderr:68911: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.104 INFO:tasks.workunit.client.0.smithi170.stderr:68911: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.104 INFO:tasks.workunit.client.0.smithi170.stderr:68911: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.16") | .last_scrub_stamp' 2023-12-01T06:32:44.104 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.104 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.104 INFO:tasks.workunit.client.0.smithi170.stderr:68911: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:44.104 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:44.238 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.5 2023-12-01T06:32:44.238 INFO:tasks.workunit.client.0.smithi170.stderr:68792: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.5 2023-12-01T06:32:44.238 INFO:tasks.workunit.client.0.smithi170.stderr:68792: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.5 2023-12-01T06:32:44.238 INFO:tasks.workunit.client.0.smithi170.stderr:68792: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.238 INFO:tasks.workunit.client.0.smithi170.stderr:68792: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.5") | .last_scrub_stamp' 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.5 query 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.5 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: instructing pg 1.5 on osd.4 to scrub 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.5") | .state' 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: dumped pgs 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.5 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.239 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.5 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:68792: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.5 last_scrub_stamp 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:68792: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.5 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:68792: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:68792: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.5") | .last_scrub_stamp' 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:68792: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:68792: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:44.240 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:44.327 INFO:tasks.workunit.client.0.smithi170.stderr:_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.327 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.328 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.328 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test true = true 2023-12-01T06:32:44.328 INFO:tasks.workunit.client.0.smithi170.stderr:68861: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:203: pg_scrub_mod: return 2 2023-12-01T06:32:44.328 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 2 2023-12-01T06:32:44.390 INFO:tasks.workunit.client.0.smithi170.stderr:_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.390 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.390 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.390 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test true = true 2023-12-01T06:32:44.390 INFO:tasks.workunit.client.0.smithi170.stderr:68791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:203: pg_scrub_mod: return 2 2023-12-01T06:32:44.390 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 2 2023-12-01T06:32:44.420 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.15 2023-12-01T06:32:44.421 INFO:tasks.workunit.client.0.smithi170.stderr:68908: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.15 2023-12-01T06:32:44.421 INFO:tasks.workunit.client.0.smithi170.stderr:68908: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.15 2023-12-01T06:32:44.421 INFO:tasks.workunit.client.0.smithi170.stderr:68908: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.421 INFO:tasks.workunit.client.0.smithi170.stderr:68908: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.421 INFO:tasks.workunit.client.0.smithi170.stderr:68908: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.15") | .last_scrub_stamp' 2023-12-01T06:32:44.421 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.421 INFO:tasks.workunit.client.0.smithi170.stderr:68908: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:44.421 INFO:tasks.workunit.client.0.smithi170.stderr:68908: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.15 query 2023-12-01T06:32:44.421 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:44.421 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:44.421 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:44.421 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.15 2023-12-01T06:32:44.421 INFO:tasks.workunit.client.0.smithi170.stderr:68908: instructing pg 1.15 on osd.1 to scrub 2023-12-01T06:32:44.422 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:44.422 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.15") | .state' 2023-12-01T06:32:44.422 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:44.422 INFO:tasks.workunit.client.0.smithi170.stderr:68908: dumped pgs 2023-12-01T06:32:44.422 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.15 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.422 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.15 2023-12-01T06:32:44.422 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.422 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:44.422 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:44.422 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:44.422 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:44.422 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:44.422 INFO:tasks.workunit.client.0.smithi170.stderr:68908: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.15 last_scrub_stamp 2023-12-01T06:32:44.422 INFO:tasks.workunit.client.0.smithi170.stderr:68908: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.15 2023-12-01T06:32:44.423 INFO:tasks.workunit.client.0.smithi170.stderr:68908: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.423 INFO:tasks.workunit.client.0.smithi170.stderr:68908: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.423 INFO:tasks.workunit.client.0.smithi170.stderr:68908: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.15") | .last_scrub_stamp' 2023-12-01T06:32:44.423 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.423 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.423 INFO:tasks.workunit.client.0.smithi170.stderr:68908: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:44.423 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:44.446 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.19 2023-12-01T06:32:44.446 INFO:tasks.workunit.client.0.smithi170.stderr:68939: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.19 2023-12-01T06:32:44.446 INFO:tasks.workunit.client.0.smithi170.stderr:68939: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.19 2023-12-01T06:32:44.446 INFO:tasks.workunit.client.0.smithi170.stderr:68939: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.447 INFO:tasks.workunit.client.0.smithi170.stderr:68939: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.447 INFO:tasks.workunit.client.0.smithi170.stderr:68939: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.19") | .last_scrub_stamp' 2023-12-01T06:32:44.447 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.447 INFO:tasks.workunit.client.0.smithi170.stderr:68939: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:44.447 INFO:tasks.workunit.client.0.smithi170.stderr:68939: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.19 query 2023-12-01T06:32:44.447 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:44.447 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:44.447 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:44.447 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.19 2023-12-01T06:32:44.447 INFO:tasks.workunit.client.0.smithi170.stderr:68939: instructing pg 1.19 on osd.3 to scrub 2023-12-01T06:32:44.447 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:44.447 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:44.447 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.19") | .state' 2023-12-01T06:32:44.447 INFO:tasks.workunit.client.0.smithi170.stderr:68939: dumped pgs 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.19 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.19 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.19 last_scrub_stamp 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.19 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.19") | .last_scrub_stamp' 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.448 INFO:tasks.workunit.client.0.smithi170.stderr:68939: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:44.449 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:44.456 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.8 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.8 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.8 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.8") | .last_scrub_stamp' 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.8 query 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.8 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: instructing pg 1.8 on osd.5 to scrub 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.8") | .state' 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: dumped pgs 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.8 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.8 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.8 last_scrub_stamp 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.8 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:68820: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.8") | .last_scrub_stamp' 2023-12-01T06:32:44.459 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.459 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.459 INFO:tasks.workunit.client.0.smithi170.stderr:68820: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:44.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:44.489 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.1d 2023-12-01T06:32:44.489 INFO:tasks.workunit.client.0.smithi170.stderr:68969: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.1d 2023-12-01T06:32:44.489 INFO:tasks.workunit.client.0.smithi170.stderr:68969: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1d 2023-12-01T06:32:44.489 INFO:tasks.workunit.client.0.smithi170.stderr:68969: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.489 INFO:tasks.workunit.client.0.smithi170.stderr:68969: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.489 INFO:tasks.workunit.client.0.smithi170.stderr:68969: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1d") | .last_scrub_stamp' 2023-12-01T06:32:44.489 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.489 INFO:tasks.workunit.client.0.smithi170.stderr:68969: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.1d query 2023-12-01T06:32:44.489 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:44.489 INFO:tasks.workunit.client.0.smithi170.stderr:68969: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:44.489 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:44.489 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.1d 2023-12-01T06:32:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:68969: instructing pg 1.1d on osd.1 to scrub 2023-12-01T06:32:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.1d") | .state' 2023-12-01T06:32:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:68969: dumped pgs 2023-12-01T06:32:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.1d 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.1d 2023-12-01T06:32:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:44.491 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:44.491 INFO:tasks.workunit.client.0.smithi170.stderr:68969: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.1d last_scrub_stamp 2023-12-01T06:32:44.491 INFO:tasks.workunit.client.0.smithi170.stderr:68969: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1d 2023-12-01T06:32:44.491 INFO:tasks.workunit.client.0.smithi170.stderr:68969: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.491 INFO:tasks.workunit.client.0.smithi170.stderr:68969: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.491 INFO:tasks.workunit.client.0.smithi170.stderr:68969: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1d") | .last_scrub_stamp' 2023-12-01T06:32:44.491 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.491 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.491 INFO:tasks.workunit.client.0.smithi170.stderr:68969: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:44.491 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:44.569 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.a 2023-12-01T06:32:44.570 INFO:tasks.workunit.client.0.smithi170.stderr:68830: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.a 2023-12-01T06:32:44.570 INFO:tasks.workunit.client.0.smithi170.stderr:68830: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.a 2023-12-01T06:32:44.570 INFO:tasks.workunit.client.0.smithi170.stderr:68830: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.570 INFO:tasks.workunit.client.0.smithi170.stderr:68830: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.a") | .last_scrub_stamp' 2023-12-01T06:32:44.570 INFO:tasks.workunit.client.0.smithi170.stderr:68830: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.570 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.570 INFO:tasks.workunit.client.0.smithi170.stderr:68830: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:44.570 INFO:tasks.workunit.client.0.smithi170.stderr:68830: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.a query 2023-12-01T06:32:44.570 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:44.570 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:44.570 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:44.570 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.a 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: instructing pg 1.a on osd.1 to scrub 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.a") | .state' 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: dumped pgs 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.a 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.a 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.a last_scrub_stamp 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.a 2023-12-01T06:32:44.571 INFO:tasks.workunit.client.0.smithi170.stderr:68830: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.572 INFO:tasks.workunit.client.0.smithi170.stderr:68830: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.a") | .last_scrub_stamp' 2023-12-01T06:32:44.572 INFO:tasks.workunit.client.0.smithi170.stderr:68830: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.572 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.572 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.572 INFO:tasks.workunit.client.0.smithi170.stderr:68830: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:44.572 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:44.584 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.0 2023-12-01T06:32:44.584 INFO:tasks.workunit.client.0.smithi170.stderr:68762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.0 2023-12-01T06:32:44.584 INFO:tasks.workunit.client.0.smithi170.stderr:68762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2023-12-01T06:32:44.584 INFO:tasks.workunit.client.0.smithi170.stderr:68762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.584 INFO:tasks.workunit.client.0.smithi170.stderr:68762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.0 query 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.0 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: instructing pg 1.0 on osd.7 to scrub 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.0") | .state' 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: dumped pgs 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.0 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.585 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.0 2023-12-01T06:32:44.586 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.586 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:44.586 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:44.586 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:44.586 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:44.586 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:44.586 INFO:tasks.workunit.client.0.smithi170.stderr:68762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.0 last_scrub_stamp 2023-12-01T06:32:44.586 INFO:tasks.workunit.client.0.smithi170.stderr:68762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2023-12-01T06:32:44.586 INFO:tasks.workunit.client.0.smithi170.stderr:68762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.586 INFO:tasks.workunit.client.0.smithi170.stderr:68762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2023-12-01T06:32:44.586 INFO:tasks.workunit.client.0.smithi170.stderr:68762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.586 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.586 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.587 INFO:tasks.workunit.client.0.smithi170.stderr:68762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:44.587 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:44.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:44.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:44.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:44.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:44.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68764 2023-12-01T06:32:44.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:44.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:44.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:44.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:44.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68771 2023-12-01T06:32:44.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:44.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:44.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:44.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:44.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68779 2023-12-01T06:32:44.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:44.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:44.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:44.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:44.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68784 2023-12-01T06:32:44.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=2 2023-12-01T06:32:44.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 2 -eq 2 2023-12-01T06:32:44.593 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:218: wait_background_check: expr 0 + 1 2023-12-01T06:32:44.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:218: wait_background_check: recov_scrub_count=1 2023-12-01T06:32:44.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:44.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68787 2023-12-01T06:32:44.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:44.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:44.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:44.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:44.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68798 2023-12-01T06:32:44.623 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.1a 2023-12-01T06:32:44.623 INFO:tasks.workunit.client.0.smithi170.stderr:68943: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.1a 2023-12-01T06:32:44.623 INFO:tasks.workunit.client.0.smithi170.stderr:68943: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1a 2023-12-01T06:32:44.623 INFO:tasks.workunit.client.0.smithi170.stderr:68943: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.623 INFO:tasks.workunit.client.0.smithi170.stderr:68943: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.623 INFO:tasks.workunit.client.0.smithi170.stderr:68943: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1a") | .last_scrub_stamp' 2023-12-01T06:32:44.623 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.623 INFO:tasks.workunit.client.0.smithi170.stderr:68943: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:44.623 INFO:tasks.workunit.client.0.smithi170.stderr:68943: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.1a query 2023-12-01T06:32:44.623 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:44.623 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:44.623 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:44.623 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.1a 2023-12-01T06:32:44.624 INFO:tasks.workunit.client.0.smithi170.stderr:68943: instructing pg 1.1a on osd.1 to scrub 2023-12-01T06:32:44.624 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:44.624 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.1a") | .state' 2023-12-01T06:32:44.624 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:44.624 INFO:tasks.workunit.client.0.smithi170.stderr:68943: dumped pgs 2023-12-01T06:32:44.624 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.1a 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.624 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.1a 2023-12-01T06:32:44.624 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.624 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:44.624 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:44.625 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:44.625 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:44.625 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:44.625 INFO:tasks.workunit.client.0.smithi170.stderr:68943: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.1a last_scrub_stamp 2023-12-01T06:32:44.625 INFO:tasks.workunit.client.0.smithi170.stderr:68943: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1a 2023-12-01T06:32:44.625 INFO:tasks.workunit.client.0.smithi170.stderr:68943: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.625 INFO:tasks.workunit.client.0.smithi170.stderr:68943: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1a") | .last_scrub_stamp' 2023-12-01T06:32:44.625 INFO:tasks.workunit.client.0.smithi170.stderr:68943: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.625 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.625 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.625 INFO:tasks.workunit.client.0.smithi170.stderr:68943: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:44.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:44.638 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.18 2023-12-01T06:32:44.638 INFO:tasks.workunit.client.0.smithi170.stderr:68929: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.18 2023-12-01T06:32:44.638 INFO:tasks.workunit.client.0.smithi170.stderr:68929: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.18 2023-12-01T06:32:44.638 INFO:tasks.workunit.client.0.smithi170.stderr:68929: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.638 INFO:tasks.workunit.client.0.smithi170.stderr:68929: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.638 INFO:tasks.workunit.client.0.smithi170.stderr:68929: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.18") | .last_scrub_stamp' 2023-12-01T06:32:44.638 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.638 INFO:tasks.workunit.client.0.smithi170.stderr:68929: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.18 query 2023-12-01T06:32:44.638 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:44.638 INFO:tasks.workunit.client.0.smithi170.stderr:68929: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:44.638 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:44.638 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:44.638 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.18 2023-12-01T06:32:44.639 INFO:tasks.workunit.client.0.smithi170.stderr:68929: instructing pg 1.18 on osd.5 to scrub 2023-12-01T06:32:44.639 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:44.639 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.18") | .state' 2023-12-01T06:32:44.639 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:44.639 INFO:tasks.workunit.client.0.smithi170.stderr:68929: dumped pgs 2023-12-01T06:32:44.639 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.18 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.639 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.18 2023-12-01T06:32:44.639 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.639 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:44.639 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:44.641 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:44.641 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:44.641 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:44.641 INFO:tasks.workunit.client.0.smithi170.stderr:68929: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.18 last_scrub_stamp 2023-12-01T06:32:44.641 INFO:tasks.workunit.client.0.smithi170.stderr:68929: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.18 2023-12-01T06:32:44.641 INFO:tasks.workunit.client.0.smithi170.stderr:68929: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.641 INFO:tasks.workunit.client.0.smithi170.stderr:68929: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.641 INFO:tasks.workunit.client.0.smithi170.stderr:68929: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.18") | .last_scrub_stamp' 2023-12-01T06:32:44.641 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.641 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.641 INFO:tasks.workunit.client.0.smithi170.stderr:68929: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:44.641 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:44.738 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.10 2023-12-01T06:32:44.738 INFO:tasks.workunit.client.0.smithi170.stderr:68869: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.10 2023-12-01T06:32:44.738 INFO:tasks.workunit.client.0.smithi170.stderr:68869: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.10 2023-12-01T06:32:44.738 INFO:tasks.workunit.client.0.smithi170.stderr:68869: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.738 INFO:tasks.workunit.client.0.smithi170.stderr:68869: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.739 INFO:tasks.workunit.client.0.smithi170.stderr:68869: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.10") | .last_scrub_stamp' 2023-12-01T06:32:44.739 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.739 INFO:tasks.workunit.client.0.smithi170.stderr:68869: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.10 query 2023-12-01T06:32:44.739 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:44.739 INFO:tasks.workunit.client.0.smithi170.stderr:68869: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:44.739 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:44.739 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:44.739 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.10 2023-12-01T06:32:44.739 INFO:tasks.workunit.client.0.smithi170.stderr:68869: instructing pg 1.10 on osd.5 to scrub 2023-12-01T06:32:44.739 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:44.739 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.10") | .state' 2023-12-01T06:32:44.739 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:44.740 INFO:tasks.workunit.client.0.smithi170.stderr:68869: dumped pgs 2023-12-01T06:32:44.740 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.10 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.740 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.10 2023-12-01T06:32:44.740 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.740 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:44.740 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:44.740 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:44.740 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:44.740 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:44.740 INFO:tasks.workunit.client.0.smithi170.stderr:68869: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.10 last_scrub_stamp 2023-12-01T06:32:44.740 INFO:tasks.workunit.client.0.smithi170.stderr:68869: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.10 2023-12-01T06:32:44.740 INFO:tasks.workunit.client.0.smithi170.stderr:68869: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.740 INFO:tasks.workunit.client.0.smithi170.stderr:68869: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.741 INFO:tasks.workunit.client.0.smithi170.stderr:68869: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.10") | .last_scrub_stamp' 2023-12-01T06:32:44.741 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.741 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.741 INFO:tasks.workunit.client.0.smithi170.stderr:68869: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:44.741 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:44.780 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.d 2023-12-01T06:32:44.780 INFO:tasks.workunit.client.0.smithi170.stderr:68855: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.d 2023-12-01T06:32:44.780 INFO:tasks.workunit.client.0.smithi170.stderr:68855: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.d 2023-12-01T06:32:44.780 INFO:tasks.workunit.client.0.smithi170.stderr:68855: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.780 INFO:tasks.workunit.client.0.smithi170.stderr:68855: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.d") | .last_scrub_stamp' 2023-12-01T06:32:44.780 INFO:tasks.workunit.client.0.smithi170.stderr:68855: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.780 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.780 INFO:tasks.workunit.client.0.smithi170.stderr:68855: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:44.780 INFO:tasks.workunit.client.0.smithi170.stderr:68855: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.d query 2023-12-01T06:32:44.780 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:44.781 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:44.781 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:44.781 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.d 2023-12-01T06:32:44.781 INFO:tasks.workunit.client.0.smithi170.stderr:68855: instructing pg 1.d on osd.7 to scrub 2023-12-01T06:32:44.781 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:44.781 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:44.781 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.d") | .state' 2023-12-01T06:32:44.781 INFO:tasks.workunit.client.0.smithi170.stderr:68855: dumped pgs 2023-12-01T06:32:44.781 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.d 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.781 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.d 2023-12-01T06:32:44.781 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.781 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:44.781 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:44.781 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:44.782 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:44.782 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:44.782 INFO:tasks.workunit.client.0.smithi170.stderr:68855: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.d last_scrub_stamp 2023-12-01T06:32:44.782 INFO:tasks.workunit.client.0.smithi170.stderr:68855: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.d 2023-12-01T06:32:44.782 INFO:tasks.workunit.client.0.smithi170.stderr:68855: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.782 INFO:tasks.workunit.client.0.smithi170.stderr:68855: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.d") | .last_scrub_stamp' 2023-12-01T06:32:44.782 INFO:tasks.workunit.client.0.smithi170.stderr:68855: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.783 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.783 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.783 INFO:tasks.workunit.client.0.smithi170.stderr:68855: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:44.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:44.941 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.b 2023-12-01T06:32:44.941 INFO:tasks.workunit.client.0.smithi170.stderr:68837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.b 2023-12-01T06:32:44.941 INFO:tasks.workunit.client.0.smithi170.stderr:68837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.b 2023-12-01T06:32:44.941 INFO:tasks.workunit.client.0.smithi170.stderr:68837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.941 INFO:tasks.workunit.client.0.smithi170.stderr:68837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.b") | .last_scrub_stamp' 2023-12-01T06:32:44.941 INFO:tasks.workunit.client.0.smithi170.stderr:68837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.941 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.941 INFO:tasks.workunit.client.0.smithi170.stderr:68837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.b query 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.b 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: instructing pg 1.b on osd.1 to scrub 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.b") | .state' 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: dumped pgs 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.b 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.b 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:44.942 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:44.943 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:44.943 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:44.943 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:44.943 INFO:tasks.workunit.client.0.smithi170.stderr:68837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.b last_scrub_stamp 2023-12-01T06:32:44.943 INFO:tasks.workunit.client.0.smithi170.stderr:68837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.b 2023-12-01T06:32:44.943 INFO:tasks.workunit.client.0.smithi170.stderr:68837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:44.943 INFO:tasks.workunit.client.0.smithi170.stderr:68837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:44.943 INFO:tasks.workunit.client.0.smithi170.stderr:68837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.b") | .last_scrub_stamp' 2023-12-01T06:32:44.943 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:44.943 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:44.943 INFO:tasks.workunit.client.0.smithi170.stderr:68837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:44.943 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:45.016 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.12 2023-12-01T06:32:45.017 INFO:tasks.workunit.client.0.smithi170.stderr:68886: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.12 2023-12-01T06:32:45.017 INFO:tasks.workunit.client.0.smithi170.stderr:68886: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.12 2023-12-01T06:32:45.017 INFO:tasks.workunit.client.0.smithi170.stderr:68886: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.017 INFO:tasks.workunit.client.0.smithi170.stderr:68886: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.017 INFO:tasks.workunit.client.0.smithi170.stderr:68886: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.12") | .last_scrub_stamp' 2023-12-01T06:32:45.017 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.017 INFO:tasks.workunit.client.0.smithi170.stderr:68886: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:45.017 INFO:tasks.workunit.client.0.smithi170.stderr:68886: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.12 query 2023-12-01T06:32:45.017 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:45.017 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:45.017 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:45.017 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.12 2023-12-01T06:32:45.017 INFO:tasks.workunit.client.0.smithi170.stderr:68886: instructing pg 1.12 on osd.0 to scrub 2023-12-01T06:32:45.017 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.12") | .state' 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: dumped pgs 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.12 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.12 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.12 last_scrub_stamp 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.12 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.018 INFO:tasks.workunit.client.0.smithi170.stderr:68886: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.019 INFO:tasks.workunit.client.0.smithi170.stderr:68886: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.12") | .last_scrub_stamp' 2023-12-01T06:32:45.019 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:45.019 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:45.019 INFO:tasks.workunit.client.0.smithi170.stderr:68886: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:45.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:45.054 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.13 2023-12-01T06:32:45.054 INFO:tasks.workunit.client.0.smithi170.stderr:68892: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.13 2023-12-01T06:32:45.054 INFO:tasks.workunit.client.0.smithi170.stderr:68892: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.13 2023-12-01T06:32:45.054 INFO:tasks.workunit.client.0.smithi170.stderr:68892: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.054 INFO:tasks.workunit.client.0.smithi170.stderr:68892: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.054 INFO:tasks.workunit.client.0.smithi170.stderr:68892: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.13") | .last_scrub_stamp' 2023-12-01T06:32:45.054 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.054 INFO:tasks.workunit.client.0.smithi170.stderr:68892: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.13 query 2023-12-01T06:32:45.054 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:45.055 INFO:tasks.workunit.client.0.smithi170.stderr:68892: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:45.055 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:45.055 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:45.055 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.13 2023-12-01T06:32:45.055 INFO:tasks.workunit.client.0.smithi170.stderr:68892: instructing pg 1.13 on osd.4 to scrub 2023-12-01T06:32:45.055 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.13") | .state' 2023-12-01T06:32:45.055 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:45.056 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:45.056 INFO:tasks.workunit.client.0.smithi170.stderr:68892: dumped pgs 2023-12-01T06:32:45.056 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.13 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.056 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.13 2023-12-01T06:32:45.056 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.056 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:45.056 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:45.057 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:45.057 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:45.057 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:45.057 INFO:tasks.workunit.client.0.smithi170.stderr:68892: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.13 last_scrub_stamp 2023-12-01T06:32:45.057 INFO:tasks.workunit.client.0.smithi170.stderr:68892: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.13 2023-12-01T06:32:45.057 INFO:tasks.workunit.client.0.smithi170.stderr:68892: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.057 INFO:tasks.workunit.client.0.smithi170.stderr:68892: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.13") | .last_scrub_stamp' 2023-12-01T06:32:45.057 INFO:tasks.workunit.client.0.smithi170.stderr:68892: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.058 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:45.058 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:45.058 INFO:tasks.workunit.client.0.smithi170.stderr:68892: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:45.058 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:45.064 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.14 2023-12-01T06:32:45.065 INFO:tasks.workunit.client.0.smithi170.stderr:68902: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.14 2023-12-01T06:32:45.065 INFO:tasks.workunit.client.0.smithi170.stderr:68902: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.14 2023-12-01T06:32:45.065 INFO:tasks.workunit.client.0.smithi170.stderr:68902: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.065 INFO:tasks.workunit.client.0.smithi170.stderr:68902: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.14") | .last_scrub_stamp' 2023-12-01T06:32:45.065 INFO:tasks.workunit.client.0.smithi170.stderr:68902: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.065 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.065 INFO:tasks.workunit.client.0.smithi170.stderr:68902: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:45.065 INFO:tasks.workunit.client.0.smithi170.stderr:68902: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.14 query 2023-12-01T06:32:45.065 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:45.065 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:45.065 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:45.065 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.14 2023-12-01T06:32:45.065 INFO:tasks.workunit.client.0.smithi170.stderr:68902: instructing pg 1.14 on osd.2 to scrub 2023-12-01T06:32:45.065 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.14") | .state' 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: dumped pgs 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.14 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.14 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.14 last_scrub_stamp 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.14 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.066 INFO:tasks.workunit.client.0.smithi170.stderr:68902: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.14") | .last_scrub_stamp' 2023-12-01T06:32:45.067 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:32:40.798713+0000 '>' '' 2023-12-01T06:32:45.067 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:45.067 INFO:tasks.workunit.client.0.smithi170.stderr:68902: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:45.067 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:45.071 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.7 2023-12-01T06:32:45.071 INFO:tasks.workunit.client.0.smithi170.stderr:68806: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.7 2023-12-01T06:32:45.071 INFO:tasks.workunit.client.0.smithi170.stderr:68806: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.7 2023-12-01T06:32:45.071 INFO:tasks.workunit.client.0.smithi170.stderr:68806: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.071 INFO:tasks.workunit.client.0.smithi170.stderr:68806: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.7") | .last_scrub_stamp' 2023-12-01T06:32:45.071 INFO:tasks.workunit.client.0.smithi170.stderr:68806: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.071 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.071 INFO:tasks.workunit.client.0.smithi170.stderr:68806: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.7 query 2023-12-01T06:32:45.071 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:45.071 INFO:tasks.workunit.client.0.smithi170.stderr:68806: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.7 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: instructing pg 1.7 on osd.6 to scrub 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.7") | .state' 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: dumped pgs 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.7 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.7 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:45.072 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:45.073 INFO:tasks.workunit.client.0.smithi170.stderr:68806: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.7 last_scrub_stamp 2023-12-01T06:32:45.073 INFO:tasks.workunit.client.0.smithi170.stderr:68806: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.7 2023-12-01T06:32:45.073 INFO:tasks.workunit.client.0.smithi170.stderr:68806: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.073 INFO:tasks.workunit.client.0.smithi170.stderr:68806: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.7") | .last_scrub_stamp' 2023-12-01T06:32:45.073 INFO:tasks.workunit.client.0.smithi170.stderr:68806: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.073 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:45.073 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:45.073 INFO:tasks.workunit.client.0.smithi170.stderr:68806: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:45.073 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:45.088 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.11 2023-12-01T06:32:45.088 INFO:tasks.workunit.client.0.smithi170.stderr:68874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.11 2023-12-01T06:32:45.088 INFO:tasks.workunit.client.0.smithi170.stderr:68874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.11 2023-12-01T06:32:45.088 INFO:tasks.workunit.client.0.smithi170.stderr:68874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.088 INFO:tasks.workunit.client.0.smithi170.stderr:68874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.11") | .last_scrub_stamp' 2023-12-01T06:32:45.088 INFO:tasks.workunit.client.0.smithi170.stderr:68874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.11 query 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.11 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: instructing pg 1.11 on osd.0 to scrub 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.11") | .state' 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: dumped pgs 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.11 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.11 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.089 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:45.090 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:45.090 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:45.090 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:45.090 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:45.090 INFO:tasks.workunit.client.0.smithi170.stderr:68874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.11 last_scrub_stamp 2023-12-01T06:32:45.090 INFO:tasks.workunit.client.0.smithi170.stderr:68874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.11 2023-12-01T06:32:45.090 INFO:tasks.workunit.client.0.smithi170.stderr:68874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.090 INFO:tasks.workunit.client.0.smithi170.stderr:68874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.090 INFO:tasks.workunit.client.0.smithi170.stderr:68874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.11") | .last_scrub_stamp' 2023-12-01T06:32:45.090 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:45.090 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:45.090 INFO:tasks.workunit.client.0.smithi170.stderr:68874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:45.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:45.103 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.f 2023-12-01T06:32:45.103 INFO:tasks.workunit.client.0.smithi170.stderr:68859: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.f 2023-12-01T06:32:45.103 INFO:tasks.workunit.client.0.smithi170.stderr:68859: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.f 2023-12-01T06:32:45.103 INFO:tasks.workunit.client.0.smithi170.stderr:68859: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.103 INFO:tasks.workunit.client.0.smithi170.stderr:68859: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.f") | .last_scrub_stamp' 2023-12-01T06:32:45.103 INFO:tasks.workunit.client.0.smithi170.stderr:68859: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.103 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.f query 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.f 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: instructing pg 1.f on osd.6 to scrub 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.f") | .state' 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: dumped pgs 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.f 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.f 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:45.104 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:45.105 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:45.105 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:45.105 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:45.105 INFO:tasks.workunit.client.0.smithi170.stderr:68859: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.f last_scrub_stamp 2023-12-01T06:32:45.105 INFO:tasks.workunit.client.0.smithi170.stderr:68859: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.f 2023-12-01T06:32:45.105 INFO:tasks.workunit.client.0.smithi170.stderr:68859: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.105 INFO:tasks.workunit.client.0.smithi170.stderr:68859: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.f") | .last_scrub_stamp' 2023-12-01T06:32:45.105 INFO:tasks.workunit.client.0.smithi170.stderr:68859: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.106 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:45.106 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:45.106 INFO:tasks.workunit.client.0.smithi170.stderr:68859: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:45.106 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:45.132 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.1b 2023-12-01T06:32:45.132 INFO:tasks.workunit.client.0.smithi170.stderr:68953: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.1b 2023-12-01T06:32:45.132 INFO:tasks.workunit.client.0.smithi170.stderr:68953: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1b 2023-12-01T06:32:45.132 INFO:tasks.workunit.client.0.smithi170.stderr:68953: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.132 INFO:tasks.workunit.client.0.smithi170.stderr:68953: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.132 INFO:tasks.workunit.client.0.smithi170.stderr:68953: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1b") | .last_scrub_stamp' 2023-12-01T06:32:45.132 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.132 INFO:tasks.workunit.client.0.smithi170.stderr:68953: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.1b query 2023-12-01T06:32:45.132 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:45.132 INFO:tasks.workunit.client.0.smithi170.stderr:68953: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:45.133 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:45.133 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:45.133 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.1b 2023-12-01T06:32:45.133 INFO:tasks.workunit.client.0.smithi170.stderr:68953: instructing pg 1.1b on osd.0 to scrub 2023-12-01T06:32:45.133 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:45.133 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:45.133 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.1b") | .state' 2023-12-01T06:32:45.133 INFO:tasks.workunit.client.0.smithi170.stderr:68953: dumped pgs 2023-12-01T06:32:45.133 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.1b 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.133 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.1b 2023-12-01T06:32:45.133 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.134 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:45.134 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:45.134 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:45.134 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:45.134 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:45.134 INFO:tasks.workunit.client.0.smithi170.stderr:68953: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.1b last_scrub_stamp 2023-12-01T06:32:45.134 INFO:tasks.workunit.client.0.smithi170.stderr:68953: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1b 2023-12-01T06:32:45.134 INFO:tasks.workunit.client.0.smithi170.stderr:68953: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.134 INFO:tasks.workunit.client.0.smithi170.stderr:68953: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1b") | .last_scrub_stamp' 2023-12-01T06:32:45.134 INFO:tasks.workunit.client.0.smithi170.stderr:68953: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.134 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:45.134 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:45.134 INFO:tasks.workunit.client.0.smithi170.stderr:68953: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:45.134 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:45.160 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.1c 2023-12-01T06:32:45.160 INFO:tasks.workunit.client.0.smithi170.stderr:68960: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.1c 2023-12-01T06:32:45.160 INFO:tasks.workunit.client.0.smithi170.stderr:68960: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1c 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1c") | .last_scrub_stamp' 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.1c query 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.1c 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: instructing pg 1.1c on osd.0 to scrub 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.1c") | .state' 2023-12-01T06:32:45.161 INFO:tasks.workunit.client.0.smithi170.stderr:68960: dumped pgs 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.1c 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.1c 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.1c last_scrub_stamp 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1c 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1c") | .last_scrub_stamp' 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:45.162 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:45.163 INFO:tasks.workunit.client.0.smithi170.stderr:68960: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:45.163 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:45.167 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.6 2023-12-01T06:32:45.167 INFO:tasks.workunit.client.0.smithi170.stderr:68801: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.6 2023-12-01T06:32:45.167 INFO:tasks.workunit.client.0.smithi170.stderr:68801: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.6 2023-12-01T06:32:45.167 INFO:tasks.workunit.client.0.smithi170.stderr:68801: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.167 INFO:tasks.workunit.client.0.smithi170.stderr:68801: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.167 INFO:tasks.workunit.client.0.smithi170.stderr:68801: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.6") | .last_scrub_stamp' 2023-12-01T06:32:45.167 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.167 INFO:tasks.workunit.client.0.smithi170.stderr:68801: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:45.167 INFO:tasks.workunit.client.0.smithi170.stderr:68801: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.6 query 2023-12-01T06:32:45.167 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:45.167 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.6 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: instructing pg 1.6 on osd.6 to scrub 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.6") | .state' 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: dumped pgs 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.6 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.6 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:45.168 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:68801: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.6 last_scrub_stamp 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:68801: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.6 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:68801: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:68801: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:68801: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.6") | .last_scrub_stamp' 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:68801: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68803 2023-12-01T06:32:45.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68815 2023-12-01T06:32:45.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68817 2023-12-01T06:32:45.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68824 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68827 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68839 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68847 2023-12-01T06:32:45.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68849 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=2 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 2 -eq 2 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:218: wait_background_check: expr 1 + 1 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:218: wait_background_check: recov_scrub_count=2 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68852 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68856 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68864 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68880 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68883 2023-12-01T06:32:45.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68893 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68899 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68905 2023-12-01T06:32:45.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.175 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.175 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.175 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.175 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68916 2023-12-01T06:32:45.238 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.1f 2023-12-01T06:32:45.238 INFO:tasks.workunit.client.0.smithi170.stderr:68993: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.1f 2023-12-01T06:32:45.238 INFO:tasks.workunit.client.0.smithi170.stderr:68993: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1f 2023-12-01T06:32:45.238 INFO:tasks.workunit.client.0.smithi170.stderr:68993: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.238 INFO:tasks.workunit.client.0.smithi170.stderr:68993: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1f") | .last_scrub_stamp' 2023-12-01T06:32:45.238 INFO:tasks.workunit.client.0.smithi170.stderr:68993: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.238 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.238 INFO:tasks.workunit.client.0.smithi170.stderr:68993: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.1f query 2023-12-01T06:32:45.238 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:45.238 INFO:tasks.workunit.client.0.smithi170.stderr:68993: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:45.238 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:45.238 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.1f 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: instructing pg 1.1f on osd.5 to scrub 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.1f") | .state' 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: dumped pgs 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.1f 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.1f 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.1f last_scrub_stamp 2023-12-01T06:32:45.239 INFO:tasks.workunit.client.0.smithi170.stderr:68993: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1f 2023-12-01T06:32:45.240 INFO:tasks.workunit.client.0.smithi170.stderr:68993: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.240 INFO:tasks.workunit.client.0.smithi170.stderr:68993: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.240 INFO:tasks.workunit.client.0.smithi170.stderr:68993: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1f") | .last_scrub_stamp' 2023-12-01T06:32:45.240 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:45.240 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:45.240 INFO:tasks.workunit.client.0.smithi170.stderr:68993: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:45.240 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:45.242 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.1e 2023-12-01T06:32:45.242 INFO:tasks.workunit.client.0.smithi170.stderr:68985: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.1e 2023-12-01T06:32:45.242 INFO:tasks.workunit.client.0.smithi170.stderr:68985: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1e 2023-12-01T06:32:45.242 INFO:tasks.workunit.client.0.smithi170.stderr:68985: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.242 INFO:tasks.workunit.client.0.smithi170.stderr:68985: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1e") | .last_scrub_stamp' 2023-12-01T06:32:45.242 INFO:tasks.workunit.client.0.smithi170.stderr:68985: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.1e query 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.1e 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: instructing pg 1.1e on osd.3 to scrub 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.1e") | .state' 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: dumped pgs 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.1e 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.1e 2023-12-01T06:32:45.243 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.244 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:45.244 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:45.244 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:45.244 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:45.244 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:45.244 INFO:tasks.workunit.client.0.smithi170.stderr:68985: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.1e last_scrub_stamp 2023-12-01T06:32:45.244 INFO:tasks.workunit.client.0.smithi170.stderr:68985: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1e 2023-12-01T06:32:45.244 INFO:tasks.workunit.client.0.smithi170.stderr:68985: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.244 INFO:tasks.workunit.client.0.smithi170.stderr:68985: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1e") | .last_scrub_stamp' 2023-12-01T06:32:45.244 INFO:tasks.workunit.client.0.smithi170.stderr:68985: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.244 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:45.244 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:45.244 INFO:tasks.workunit.client.0.smithi170.stderr:68985: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:45.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:45.280 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:189: pg_scrub_mod: local pgid=1.17 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: get_last_scrub_stamp 1.17 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.17 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.17") | .last_scrub_stamp' 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:190: pg_scrub_mod: local last_scrub=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: bin/ceph pg 1.17 query 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh: line 192: bin/ceph: No such file or directory 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: jq '.acting[0]' 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:192: pg_scrub_mod: local my_primary= 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:193: pg_scrub_mod: local recovery=false 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:194: pg_scrub_mod: ceph pg scrub 1.17 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: instructing pg 1.17 on osd.0 to scrub 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: grep -q recovering 2023-12-01T06:32:45.281 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: ceph --format json pg dump pgs 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:196: pg_scrub_mod: jq '.pg_stats | .[] | select(.pgid == "1.17") | .state' 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: dumped pgs 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:200: pg_scrub_mod: wait_for_scrub_mod 1.17 2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:154: wait_for_scrub_mod: local pgid=1.17 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:155: wait_for_scrub_mod: local orig_primary=2023-12-01T06:31:13.087675+0000 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:156: wait_for_scrub_mod: local last_scrub= 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:157: wait_for_scrub_mod: local sname=last_scrub_stamp 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i=0 )) 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:159: wait_for_scrub_mod: (( i < 300 )) 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:160: wait_for_scrub_mod: sleep 0.2 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: //home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: get_last_scrub_stamp 1.17 last_scrub_stamp 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.17 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.17") | .last_scrub_stamp' 2023-12-01T06:32:45.282 INFO:tasks.workunit.client.0.smithi170.stderr:68920: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:161: wait_for_scrub_mod: test 2023-12-01T06:31:13.087675+0000 '>' '' 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:162: wait_for_scrub_mod: return 0 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:68920: /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:201: pg_scrub_mod: test false = true 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: run_in_background: return 0 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68925 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68930 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68935 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68944 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68954 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68966 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68975 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:213: wait_background_check: for pid in $pids 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:214: wait_background_check: wait 68983 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:215: wait_background_check: retcode=0 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:216: wait_background_check: test 0 -eq 2 2023-12-01T06:32:45.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:219: wait_background_check: test 0 -ne 0 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:227: wait_background_check: eval 'pids='\'''\''' 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:227: wait_background_check: pids= 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:229: wait_background_check: return 0 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:307: TEST_recovery_scrub_2: return_code=0 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:308: TEST_recovery_scrub_2: '[' 0 -ne 0 ']' 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:310: TEST_recovery_scrub_2: ERRORS=0 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:311: TEST_recovery_scrub_2: test 2 -eq 0 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:317: TEST_recovery_scrub_2: find td/osd-recovery-scrub 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:317: TEST_recovery_scrub_2: grep '[^/]*\.pid' 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:317: TEST_recovery_scrub_2: pidfile='td/osd-recovery-scrub/osd.5.pid 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr:td/osd-recovery-scrub/osd.2.pid 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr:td/osd-recovery-scrub/osd.3.pid 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr:td/osd-recovery-scrub/mgr.x.pid 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr:td/osd-recovery-scrub/osd.7.pid 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr:td/osd-recovery-scrub/osd.4.pid 2023-12-01T06:32:45.286 INFO:tasks.workunit.client.0.smithi170.stderr:td/osd-recovery-scrub/mon.a.pid 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:td/osd-recovery-scrub/osd.1.pid 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:td/osd-recovery-scrub/osd.0.pid 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:td/osd-recovery-scrub/osd.6.pid' 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:318: TEST_recovery_scrub_2: cat td/osd-recovery-scrub/osd.5.pid td/osd-recovery-scrub/osd.2.pid td/osd-recovery-scrub/osd.3.pid td/osd-recovery-scrub/mgr.x.pid td/osd-recovery-scrub/osd.7.pid td/osd-recovery-scrub/osd.4.pid td/osd-recovery-scrub/mon.a.pid td/osd-recovery-scrub/osd.1.pid td/osd-recovery-scrub/osd.0.pid td/osd-recovery-scrub/osd.6.pid 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:318: TEST_recovery_scrub_2: pid='63790 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:59662 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:61038 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:55967 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:66542 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:62414 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:55883 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:58286 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:56910 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:65166' 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:319: TEST_recovery_scrub_2: kill -0 63790 59662 61038 55967 66542 62414 55883 58286 56910 65166 2023-12-01T06:32:45.287 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:327: TEST_recovery_scrub_2: kill_daemons td/osd-recovery-scrub 2023-12-01T06:32:45.288 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-12-01T06:32:45.288 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-12-01T06:32:45.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-12-01T06:32:45.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-12-01T06:32:45.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-12-01T06:32:50.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-12-01T06:32:50.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:329: TEST_recovery_scrub_2: declare -a err_strings 2023-12-01T06:32:50.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:330: TEST_recovery_scrub_2: err_strings[0]='not scheduling scrubs due to active recovery' 2023-12-01T06:32:50.597 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:332: TEST_recovery_scrub_2: expr 8 - 1 2023-12-01T06:32:50.598 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:332: TEST_recovery_scrub_2: seq 0 7 2023-12-01T06:32:50.599 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:332: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.599 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:334: TEST_recovery_scrub_2: grep 'not scheduling scrubs' td/osd-recovery-scrub/osd.0.log 2023-12-01T06:32:50.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:332: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:334: TEST_recovery_scrub_2: grep 'not scheduling scrubs' td/osd-recovery-scrub/osd.1.log 2023-12-01T06:32:50.611 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:332: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.611 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:334: TEST_recovery_scrub_2: grep 'not scheduling scrubs' td/osd-recovery-scrub/osd.2.log 2023-12-01T06:32:50.614 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:332: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:334: TEST_recovery_scrub_2: grep 'not scheduling scrubs' td/osd-recovery-scrub/osd.3.log 2023-12-01T06:32:50.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:332: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:334: TEST_recovery_scrub_2: grep 'not scheduling scrubs' td/osd-recovery-scrub/osd.4.log 2023-12-01T06:32:50.624 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:332: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.625 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:334: TEST_recovery_scrub_2: grep 'not scheduling scrubs' td/osd-recovery-scrub/osd.5.log 2023-12-01T06:32:50.629 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:332: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.629 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:334: TEST_recovery_scrub_2: grep 'not scheduling scrubs' td/osd-recovery-scrub/osd.6.log 2023-12-01T06:32:50.633 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:332: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.633 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:334: TEST_recovery_scrub_2: grep 'not scheduling scrubs' td/osd-recovery-scrub/osd.7.log 2023-12-01T06:32:50.637 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:336: TEST_recovery_scrub_2: for err_string in "${err_strings[@]}" 2023-12-01T06:32:50.637 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:338: TEST_recovery_scrub_2: found=false 2023-12-01T06:32:50.638 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:339: TEST_recovery_scrub_2: expr 8 - 1 2023-12-01T06:32:50.639 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:339: TEST_recovery_scrub_2: seq 0 7 2023-12-01T06:32:50.640 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:339: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.640 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:341: TEST_recovery_scrub_2: grep 'not scheduling scrubs due to active recovery' td/osd-recovery-scrub/osd.0.log 2023-12-01T06:32:50.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:339: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:341: TEST_recovery_scrub_2: grep 'not scheduling scrubs due to active recovery' td/osd-recovery-scrub/osd.1.log 2023-12-01T06:32:50.649 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:339: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.649 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:341: TEST_recovery_scrub_2: grep 'not scheduling scrubs due to active recovery' td/osd-recovery-scrub/osd.2.log 2023-12-01T06:32:50.652 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:339: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:341: TEST_recovery_scrub_2: grep 'not scheduling scrubs due to active recovery' td/osd-recovery-scrub/osd.3.log 2023-12-01T06:32:50.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:339: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:341: TEST_recovery_scrub_2: grep 'not scheduling scrubs due to active recovery' td/osd-recovery-scrub/osd.4.log 2023-12-01T06:32:50.661 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:339: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.661 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:341: TEST_recovery_scrub_2: grep 'not scheduling scrubs due to active recovery' td/osd-recovery-scrub/osd.5.log 2023-12-01T06:32:50.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:339: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:341: TEST_recovery_scrub_2: grep 'not scheduling scrubs due to active recovery' td/osd-recovery-scrub/osd.6.log 2023-12-01T06:32:50.669 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:339: TEST_recovery_scrub_2: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:50.669 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:341: TEST_recovery_scrub_2: grep 'not scheduling scrubs due to active recovery' td/osd-recovery-scrub/osd.7.log 2023-12-01T06:32:50.672 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:346: TEST_recovery_scrub_2: '[' false = true ']' 2023-12-01T06:32:50.672 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:352: TEST_recovery_scrub_2: teardown td/osd-recovery-scrub 2023-12-01T06:32:50.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-recovery-scrub 2023-12-01T06:32:50.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-12-01T06:32:50.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-recovery-scrub KILL 2023-12-01T06:32:50.673 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-12-01T06:32:50.673 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-12-01T06:32:50.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-12-01T06:32:50.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-12-01T06:32:50.674 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-12-01T06:32:50.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-12-01T06:32:50.684 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-12-01T06:32:50.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-12-01T06:32:50.685 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-12-01T06:32:50.686 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-12-01T06:32:50.686 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-12-01T06:32:50.686 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-12-01T06:32:50.687 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:32:50.687 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-12-01T06:32:50.688 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-12-01T06:32:50.688 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:32:50.688 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-12-01T06:32:50.689 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-12-01T06:32:50.689 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-recovery-scrub 2023-12-01T06:32:50.716 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-12-01T06:32:50.716 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:32:50.716 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:32:50.716 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.47198 2023-12-01T06:32:50.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-12-01T06:32:50.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-12-01T06:32:50.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:354: TEST_recovery_scrub_2: '[' 0 '!=' 0 ']' 2023-12-01T06:32:50.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:360: TEST_recovery_scrub_2: echo 'TEST PASSED' 2023-12-01T06:32:50.717 INFO:tasks.workunit.client.0.smithi170.stdout:TEST PASSED 2023-12-01T06:32:50.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:361: TEST_recovery_scrub_2: return 0 2023-12-01T06:32:50.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2314: main: code=0 2023-12-01T06:32:50.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2318: main: teardown td/osd-recovery-scrub 0 2023-12-01T06:32:50.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-recovery-scrub 2023-12-01T06:32:50.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2023-12-01T06:32:50.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-recovery-scrub KILL 2023-12-01T06:32:50.719 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-12-01T06:32:50.719 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-12-01T06:32:50.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-12-01T06:32:50.720 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-12-01T06:32:50.720 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-12-01T06:32:50.720 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-12-01T06:32:50.720 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-12-01T06:32:50.721 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-12-01T06:32:50.721 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-12-01T06:32:50.721 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-12-01T06:32:50.721 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-12-01T06:32:50.722 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-12-01T06:32:50.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:32:50.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-12-01T06:32:50.723 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-12-01T06:32:50.723 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:32:50.724 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-12-01T06:32:50.724 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2023-12-01T06:32:50.725 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-recovery-scrub 2023-12-01T06:32:50.726 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-12-01T06:32:50.726 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:32:50.726 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.47198 2023-12-01T06:32:50.726 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.47198 2023-12-01T06:32:50.727 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-12-01T06:32:50.727 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-12-01T06:32:50.727 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2319: main: return 0 2023-12-01T06:32:50.728 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-01T06:32:50.728 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-01T06:32:50.782 INFO:tasks.workunit:Running workunit scrub/osd-scrub-dump.sh... 2023-12-01T06:32:50.785 DEBUG:teuthology.orchestra.run.smithi170:workunit test scrub/osd-scrub-dump.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bbeb098e76b9019fafbe1c7bd55e48604c324c89 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh 2023-12-01T06:32:50.834 INFO:tasks.workunit.client.0.smithi170.stderr:stty: 'standard input': Inappropriate ioctl for device 2023-12-01T06:32:50.839 INFO:tasks.workunit.client.0.smithi170.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2023-12-01T06:32:50.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2307: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2023-12-01T06:32:50.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2307: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2023-12-01T06:32:50.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2308: main: export PYTHONWARNINGS=ignore 2023-12-01T06:32:50.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2308: main: PYTHONWARNINGS=ignore 2023-12-01T06:32:50.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: main: export CEPH_CONF=/dev/null 2023-12-01T06:32:50.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: main: CEPH_CONF=/dev/null 2023-12-01T06:32:50.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2310: main: unset CEPH_ARGS 2023-12-01T06:32:50.840 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2312: main: local code 2023-12-01T06:32:50.840 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2313: main: run td/osd-scrub-dump 2023-12-01T06:32:50.840 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:25: run: local dir=td/osd-scrub-dump 2023-12-01T06:32:50.840 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:26: run: shift 2023-12-01T06:32:50.840 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:27: run: local CHUNK_MAX=5 2023-12-01T06:32:50.840 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:29: run: export CEPH_MON=127.0.0.1:7184 2023-12-01T06:32:50.840 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:29: run: CEPH_MON=127.0.0.1:7184 2023-12-01T06:32:50.840 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:30: run: export CEPH_ARGS 2023-12-01T06:32:50.840 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:31: run: uuidgen 2023-12-01T06:32:50.841 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:31: run: CEPH_ARGS+='--fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none ' 2023-12-01T06:32:50.841 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:32: run: CEPH_ARGS+='--mon-host=127.0.0.1:7184 ' 2023-12-01T06:32:50.841 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:33: run: CEPH_ARGS+='--osd_max_scrubs=4 ' 2023-12-01T06:32:50.841 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:34: run: CEPH_ARGS+='--osd_shallow_scrub_chunk_max=5 ' 2023-12-01T06:32:50.841 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:35: run: CEPH_ARGS+='--osd_scrub_sleep=3 ' 2023-12-01T06:32:50.841 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:36: run: CEPH_ARGS+='--osd_pool_default_size=3 ' 2023-12-01T06:32:50.841 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:40: run: CEPH_ARGS+='--osd_op_queue=wpq ' 2023-12-01T06:32:50.841 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:42: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2023-12-01T06:32:50.841 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:43: run: set 2023-12-01T06:32:50.841 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:43: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2023-12-01T06:32:50.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:43: run: local funcs=TEST_recover_unexpected 2023-12-01T06:32:50.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:44: run: for func in $funcs 2023-12-01T06:32:50.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:45: run: setup td/osd-scrub-dump 2023-12-01T06:32:50.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-dump 2023-12-01T06:32:50.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-dump 2023-12-01T06:32:50.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-dump 2023-12-01T06:32:50.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-12-01T06:32:50.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-dump KILL 2023-12-01T06:32:50.843 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-12-01T06:32:50.843 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-12-01T06:32:50.844 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-12-01T06:32:50.844 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-12-01T06:32:50.844 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-12-01T06:32:50.845 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-12-01T06:32:50.845 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-12-01T06:32:50.846 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-12-01T06:32:50.846 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-12-01T06:32:50.847 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-12-01T06:32:50.847 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-12-01T06:32:50.848 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-12-01T06:32:50.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:32:50.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-12-01T06:32:50.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-12-01T06:32:50.849 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:32:50.850 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-12-01T06:32:50.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-12-01T06:32:50.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-dump 2023-12-01T06:32:50.852 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-12-01T06:32:50.852 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:32:50.852 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:32:50.852 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.72213 2023-12-01T06:32:50.853 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-12-01T06:32:50.853 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-12-01T06:32:50.853 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-dump 2023-12-01T06:32:50.855 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2023-12-01T06:32:50.855 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:32:50.855 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:32:50.855 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.72213 2023-12-01T06:32:50.857 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2023-12-01T06:32:50.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2023-12-01T06:32:50.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2023-12-01T06:32:50.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-dump 1' TERM HUP INT 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:46: run: TEST_recover_unexpected td/osd-scrub-dump 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:52: TEST_recover_unexpected: local dir=td/osd-scrub-dump 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:53: TEST_recover_unexpected: shift 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:54: TEST_recover_unexpected: local OSDS=6 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:55: TEST_recover_unexpected: local PGS=16 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:56: TEST_recover_unexpected: local POOLS=3 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:57: TEST_recover_unexpected: local OBJS=1000 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:59: TEST_recover_unexpected: run_mon td/osd-scrub-dump a 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-dump 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-dump/a 2023-12-01T06:32:50.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-dump/a --run-dir=td/osd-scrub-dump 2023-12-01T06:32:51.249 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2023-12-01T06:32:51.249 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:32:51.250 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:32:51.250 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:32:51.250 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:32:51.250 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:32:51.250 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.72213/$cluster-$name.asok' 2023-12-01T06:32:51.251 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-dump/a '--log-file=td/osd-scrub-dump/$name.log' '--admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-dump/log --run-dir=td/osd-scrub-dump '--pid-file=td/osd-scrub-dump/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2023-12-01T06:32:51.486 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2023-12-01T06:32:51.486 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2023-12-01T06:32:51.486 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2023-12-01T06:32:51.486 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2023-12-01T06:32:51.487 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2023-12-01T06:32:51.487 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2023-12-01T06:32:51.488 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-12-01T06:32:51.488 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-12-01T06:32:51.488 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2023-12-01T06:32:51.488 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:32:51.488 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:32:51.488 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:32:51.488 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-mon.a.asok 2023-12-01T06:32:51.488 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2023-12-01T06:32:51.489 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.72213/ceph-mon.a.asok config get fsid 2023-12-01T06:32:51.603 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2023-12-01T06:32:51.603 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2023-12-01T06:32:51.603 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2023-12-01T06:32:51.603 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2023-12-01T06:32:51.603 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2023-12-01T06:32:51.603 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2023-12-01T06:32:51.603 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-12-01T06:32:51.603 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-12-01T06:32:51.603 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:32:51.604 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:32:51.604 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:32:51.604 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-mon.a.asok 2023-12-01T06:32:51.604 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2023-12-01T06:32:51.604 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.72213/ceph-mon.a.asok config get mon_host 2023-12-01T06:32:51.703 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:60: TEST_recover_unexpected: run_mgr td/osd-scrub-dump x 2023-12-01T06:32:51.703 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-dump 2023-12-01T06:32:51.703 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2023-12-01T06:32:51.703 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2023-12-01T06:32:51.703 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2023-12-01T06:32:51.704 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-dump/x 2023-12-01T06:32:51.704 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2023-12-01T06:32:52.569 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2023-12-01T06:32:52.570 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:32:52.570 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:32:52.570 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:32:52.570 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:32:52.570 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:32:52.570 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.72213/$cluster-$name.asok' 2023-12-01T06:32:52.570 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-12-01T06:32:52.571 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-dump/x '--log-file=td/osd-scrub-dump/$name.log' '--admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' --run-dir=td/osd-scrub-dump '--pid-file=td/osd-scrub-dump/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-12-01T06:32:52.610 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:61: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:32:52.611 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:61: TEST_recover_unexpected: seq 0 5 2023-12-01T06:32:52.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:61: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:32:52.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:63: TEST_recover_unexpected: run_osd td/osd-scrub-dump 0 2023-12-01T06:32:52.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-dump 2023-12-01T06:32:52.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:32:52.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2023-12-01T06:32:52.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:32:52.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-dump/0 2023-12-01T06:32:52.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq ' 2023-12-01T06:32:52.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:32:52.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:32:52.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:32:52.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-dump/0' 2023-12-01T06:32:52.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-dump/0/journal' 2023-12-01T06:32:52.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:32:52.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:32:52.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-dump' 2023-12-01T06:32:52.617 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:32:52.618 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:32:52.618 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:32:52.618 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:32:52.618 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:32:52.618 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:32:52.618 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.72213/$cluster-$name.asok' 2023-12-01T06:32:52.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' 2023-12-01T06:32:52.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:32:52.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:32:52.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:32:52.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-dump/$name.log' 2023-12-01T06:32:52.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-dump/$name.pid' 2023-12-01T06:32:52.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:32:52.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:32:52.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:32:52.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:32:52.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:32:52.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2023-12-01T06:32:52.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-dump/0 2023-12-01T06:32:52.619 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:32:52.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=7cce7099-fc22-4af5-bfeb-bfc6930d2e92 2023-12-01T06:32:52.619 INFO:tasks.workunit.client.0.smithi170.stdout:add osd0 7cce7099-fc22-4af5-bfeb-bfc6930d2e92 2023-12-01T06:32:52.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 7cce7099-fc22-4af5-bfeb-bfc6930d2e92' 2023-12-01T06:32:52.620 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:32:52.638 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAUfmllmw47JhAAnUnFOjADJ+bWMP/2oyLgeA== 2023-12-01T06:32:52.638 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAUfmllmw47JhAAnUnFOjADJ+bWMP/2oyLgeA=="}' 2023-12-01T06:32:52.638 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 7cce7099-fc22-4af5-bfeb-bfc6930d2e92 -i td/osd-scrub-dump/0/new.json 2023-12-01T06:32:52.881 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:32:52.898 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-dump/0/new.json 2023-12-01T06:32:52.898 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-dump/0 --osd-journal=td/osd-scrub-dump/0/journal --chdir= --run-dir=td/osd-scrub-dump '--admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-dump/$name.log' '--pid-file=td/osd-scrub-dump/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAUfmllmw47JhAAnUnFOjADJ+bWMP/2oyLgeA== --osd-uuid 7cce7099-fc22-4af5-bfeb-bfc6930d2e92 2023-12-01T06:32:52.937 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:32:52.940+0000 7fe8716f1640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:32:52.940 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:32:52.940+0000 7fe8716f1640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:32:52.940 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:32:52.944+0000 7fe8716f1640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:32:52.942 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:32:52.944+0000 7fe8716f1640 -1 bluestore(td/osd-scrub-dump/0/block) _read_bdev_label failed to open td/osd-scrub-dump/0/block: (2) No such file or directory 2023-12-01T06:32:52.942 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:32:52.944+0000 7fe8716f1640 -1 bluestore(td/osd-scrub-dump/0/block) _read_bdev_label failed to open td/osd-scrub-dump/0/block: (2) No such file or directory 2023-12-01T06:32:52.942 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:32:52.944+0000 7fe8716f1640 -1 bluestore(td/osd-scrub-dump/0/block) _read_bdev_label failed to open td/osd-scrub-dump/0/block: (2) No such file or directory 2023-12-01T06:32:53.024 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:32:53.024+0000 7fe8716f1640 -1 bluestore(td/osd-scrub-dump/0) _read_fsid unparsable uuid 2023-12-01T06:32:59.047 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-dump/0/keyring 2023-12-01T06:32:59.047 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:32:59.049 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd0 key to auth repository 2023-12-01T06:32:59.049 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2023-12-01T06:32:59.049 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-dump/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:32:59.443 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.0 2023-12-01T06:32:59.443 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2023-12-01T06:32:59.443 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:32:59.444 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-dump/0 --osd-journal=td/osd-scrub-dump/0/journal --chdir= --run-dir=td/osd-scrub-dump '--admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-dump/$name.log' '--pid-file=td/osd-scrub-dump/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2023-12-01T06:32:59.444 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:32:59.444 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:32:59.484 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:32:59.488+0000 7f70b6502640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:32:59.498 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:32:59.500+0000 7f70b6502640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:32:59.499 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:32:59.500+0000 7f70b6502640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:32:59.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2023-12-01T06:32:59.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:32:59.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2023-12-01T06:32:59.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:32:59.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:32:59.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:32:59.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:32:59.762 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:32:59.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:32:59.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:33:00.041 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:00.044+0000 7f70b6502640 -1 Falling back to public interface 2023-12-01T06:33:00.067 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:01.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:01.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:01.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:33:01.069 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:33:01.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:01.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:33:01.391 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:02.393 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:02.393 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:02.393 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:33:02.394 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:33:02.394 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:02.394 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:33:02.577 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:02.580+0000 7f70b6502640 -1 osd.0 0 log_to_monitors true 2023-12-01T06:33:02.748 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:03.749 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:03.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:03.750 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:33:03.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:33:03.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:03.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2023-12-01T06:33:04.088 INFO:tasks.workunit.client.0.smithi170.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3856243111,v1:127.0.0.1:6803/3856243111] [v2:127.0.0.1:6804/3856243111,v1:127.0.0.1:6805/3856243111] exists,up 7cce7099-fc22-4af5-bfeb-bfc6930d2e92 2023-12-01T06:33:04.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:61: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:63: TEST_recover_unexpected: run_osd td/osd-scrub-dump 1 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-dump 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-dump/1 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq ' 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-dump/1' 2023-12-01T06:33:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-dump/1/journal' 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-dump' 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.72213/$cluster-$name.asok' 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:33:04.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-dump/$name.log' 2023-12-01T06:33:04.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-dump/$name.pid' 2023-12-01T06:33:04.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:33:04.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:33:04.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:33:04.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:33:04.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:33:04.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2023-12-01T06:33:04.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-dump/1 2023-12-01T06:33:04.093 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:33:04.094 INFO:tasks.workunit.client.0.smithi170.stdout:add osd1 505b7d19-d5af-4567-b26e-a980a1252b47 2023-12-01T06:33:04.094 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=505b7d19-d5af-4567-b26e-a980a1252b47 2023-12-01T06:33:04.094 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 505b7d19-d5af-4567-b26e-a980a1252b47' 2023-12-01T06:33:04.094 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:33:04.116 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAgfmll5qknBxAAhhP0AvGIfFgeD9vjNyuM+g== 2023-12-01T06:33:04.116 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAgfmll5qknBxAAhhP0AvGIfFgeD9vjNyuM+g=="}' 2023-12-01T06:33:04.116 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 505b7d19-d5af-4567-b26e-a980a1252b47 -i td/osd-scrub-dump/1/new.json 2023-12-01T06:33:04.486 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:33:04.502 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-dump/1/new.json 2023-12-01T06:33:04.504 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-dump/1 --osd-journal=td/osd-scrub-dump/1/journal --chdir= --run-dir=td/osd-scrub-dump '--admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-dump/$name.log' '--pid-file=td/osd-scrub-dump/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAgfmll5qknBxAAhhP0AvGIfFgeD9vjNyuM+g== --osd-uuid 505b7d19-d5af-4567-b26e-a980a1252b47 2023-12-01T06:33:04.546 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:04.548+0000 7f4d9e8db640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:04.570 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:04.572+0000 7f4d9e8db640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:04.571 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:04.572+0000 7f4d9e8db640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:04.572 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:04.576+0000 7f4d9e8db640 -1 bluestore(td/osd-scrub-dump/1/block) _read_bdev_label failed to open td/osd-scrub-dump/1/block: (2) No such file or directory 2023-12-01T06:33:04.572 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:04.576+0000 7f4d9e8db640 -1 bluestore(td/osd-scrub-dump/1/block) _read_bdev_label failed to open td/osd-scrub-dump/1/block: (2) No such file or directory 2023-12-01T06:33:04.572 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:04.576+0000 7f4d9e8db640 -1 bluestore(td/osd-scrub-dump/1/block) _read_bdev_label failed to open td/osd-scrub-dump/1/block: (2) No such file or directory 2023-12-01T06:33:04.644 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:04.648+0000 7f4d9e8db640 -1 bluestore(td/osd-scrub-dump/1) _read_fsid unparsable uuid 2023-12-01T06:33:11.040 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-dump/1/keyring 2023-12-01T06:33:11.040 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:33:11.041 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd1 key to auth repository 2023-12-01T06:33:11.042 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2023-12-01T06:33:11.042 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-dump/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:33:11.441 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.1 2023-12-01T06:33:11.442 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2023-12-01T06:33:11.442 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-dump/1 --osd-journal=td/osd-scrub-dump/1/journal --chdir= --run-dir=td/osd-scrub-dump '--admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-dump/$name.log' '--pid-file=td/osd-scrub-dump/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2023-12-01T06:33:11.442 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:33:11.442 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:33:11.442 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:33:11.483 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:11.484+0000 7f6345548640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:11.501 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:11.504+0000 7f6345548640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:11.502 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:11.504+0000 7f6345548640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:11.766 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2023-12-01T06:33:11.766 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:33:11.767 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2023-12-01T06:33:11.767 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:33:11.767 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:33:11.767 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:11.767 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:33:11.767 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:33:11.767 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:11.767 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:33:12.049 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:12.052+0000 7f6345548640 -1 Falling back to public interface 2023-12-01T06:33:12.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:13.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:13.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:13.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:33:13.093 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:33:13.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:13.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:33:13.417 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:14.419 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:14.419 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:14.420 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:33:14.420 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:33:14.420 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:14.420 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:33:14.556 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:14.560+0000 7f6345548640 -1 osd.1 0 log_to_monitors true 2023-12-01T06:33:14.760 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:15.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:15.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:15.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:33:15.762 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:33:15.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:15.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2023-12-01T06:33:16.085 INFO:tasks.workunit.client.0.smithi170.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/306817614,v1:127.0.0.1:6811/306817614] [v2:127.0.0.1:6812/306817614,v1:127.0.0.1:6813/306817614] exists,up 505b7d19-d5af-4567-b26e-a980a1252b47 2023-12-01T06:33:16.088 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:33:16.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:33:16.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:33:16.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:61: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:33:16.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:63: TEST_recover_unexpected: run_osd td/osd-scrub-dump 2 2023-12-01T06:33:16.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-dump 2023-12-01T06:33:16.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:33:16.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2023-12-01T06:33:16.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:33:16.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-dump/2 2023-12-01T06:33:16.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq ' 2023-12-01T06:33:16.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:33:16.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:33:16.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:33:16.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-dump/2' 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-dump/2/journal' 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-dump' 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.72213/$cluster-$name.asok' 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:33:16.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:33:16.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-dump/$name.log' 2023-12-01T06:33:16.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-dump/$name.pid' 2023-12-01T06:33:16.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:33:16.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:33:16.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:33:16.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:33:16.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:33:16.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2023-12-01T06:33:16.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-dump/2 2023-12-01T06:33:16.092 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:33:16.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=6c249fd2-0c41-4bcd-a32c-81c84101aaa0 2023-12-01T06:33:16.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 6c249fd2-0c41-4bcd-a32c-81c84101aaa0' 2023-12-01T06:33:16.092 INFO:tasks.workunit.client.0.smithi170.stdout:add osd2 6c249fd2-0c41-4bcd-a32c-81c84101aaa0 2023-12-01T06:33:16.092 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:33:16.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAsfmllWa+TBxAAgyfOuv+t2snn5TCN/C3S1A== 2023-12-01T06:33:16.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAsfmllWa+TBxAAgyfOuv+t2snn5TCN/C3S1A=="}' 2023-12-01T06:33:16.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 6c249fd2-0c41-4bcd-a32c-81c84101aaa0 -i td/osd-scrub-dump/2/new.json 2023-12-01T06:33:16.491 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:33:16.508 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-dump/2/new.json 2023-12-01T06:33:16.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-dump/2 --osd-journal=td/osd-scrub-dump/2/journal --chdir= --run-dir=td/osd-scrub-dump '--admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-dump/$name.log' '--pid-file=td/osd-scrub-dump/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAsfmllWa+TBxAAgyfOuv+t2snn5TCN/C3S1A== --osd-uuid 6c249fd2-0c41-4bcd-a32c-81c84101aaa0 2023-12-01T06:33:16.548 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:16.551+0000 7f2286273640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:16.592 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:16.595+0000 7f2286273640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:16.594 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:16.595+0000 7f2286273640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:16.594 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:16.595+0000 7f2286273640 -1 bluestore(td/osd-scrub-dump/2/block) _read_bdev_label failed to open td/osd-scrub-dump/2/block: (2) No such file or directory 2023-12-01T06:33:16.595 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:16.595+0000 7f2286273640 -1 bluestore(td/osd-scrub-dump/2/block) _read_bdev_label failed to open td/osd-scrub-dump/2/block: (2) No such file or directory 2023-12-01T06:33:16.595 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:16.595+0000 7f2286273640 -1 bluestore(td/osd-scrub-dump/2/block) _read_bdev_label failed to open td/osd-scrub-dump/2/block: (2) No such file or directory 2023-12-01T06:33:16.657 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:16.659+0000 7f2286273640 -1 bluestore(td/osd-scrub-dump/2) _read_fsid unparsable uuid 2023-12-01T06:33:23.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-dump/2/keyring 2023-12-01T06:33:23.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:33:23.250 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd2 key to auth repository 2023-12-01T06:33:23.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2023-12-01T06:33:23.251 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-dump/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:33:23.639 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.2 2023-12-01T06:33:23.639 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2023-12-01T06:33:23.639 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-dump/2 --osd-journal=td/osd-scrub-dump/2/journal --chdir= --run-dir=td/osd-scrub-dump '--admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-dump/$name.log' '--pid-file=td/osd-scrub-dump/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2023-12-01T06:33:23.640 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:33:23.640 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:33:23.641 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:33:23.680 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:23.683+0000 7f5af84a2640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:23.694 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:23.695+0000 7f5af84a2640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:23.695 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:23.695+0000 7f5af84a2640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:23.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2023-12-01T06:33:23.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:33:23.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2023-12-01T06:33:23.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:33:23.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:33:23.962 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:23.962 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:33:23.962 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:33:23.962 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:23.962 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:33:24.237 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:24.239+0000 7f5af84a2640 -1 Falling back to public interface 2023-12-01T06:33:24.280 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:25.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:25.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:25.281 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:33:25.282 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:33:25.282 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:25.282 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:33:25.591 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:26.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:26.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:26.593 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:33:26.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:33:26.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:26.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:33:26.763 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:26.763+0000 7f5af84a2640 -1 osd.2 0 log_to_monitors true 2023-12-01T06:33:26.943 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:27.944 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:27.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:27.945 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:33:27.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:33:27.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:27.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2023-12-01T06:33:28.271 INFO:tasks.workunit.client.0.smithi170.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/974602583,v1:127.0.0.1:6819/974602583] [v2:127.0.0.1:6820/974602583,v1:127.0.0.1:6821/974602583] exists,up 6c249fd2-0c41-4bcd-a32c-81c84101aaa0 2023-12-01T06:33:28.275 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:33:28.275 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:33:28.275 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:33:28.275 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:61: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:33:28.275 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:63: TEST_recover_unexpected: run_osd td/osd-scrub-dump 3 2023-12-01T06:33:28.275 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-dump 2023-12-01T06:33:28.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:33:28.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2023-12-01T06:33:28.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:33:28.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-dump/3 2023-12-01T06:33:28.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq ' 2023-12-01T06:33:28.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:33:28.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:33:28.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:33:28.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-dump/3' 2023-12-01T06:33:28.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-dump/3/journal' 2023-12-01T06:33:28.277 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:33:28.277 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:33:28.277 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-dump' 2023-12-01T06:33:28.277 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:33:28.277 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:33:28.277 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:33:28.277 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:33:28.277 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:33:28.278 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:33:28.278 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.72213/$cluster-$name.asok' 2023-12-01T06:33:28.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' 2023-12-01T06:33:28.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:33:28.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:33:28.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:33:28.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-dump/$name.log' 2023-12-01T06:33:28.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-dump/$name.pid' 2023-12-01T06:33:28.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:33:28.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:33:28.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:33:28.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:33:28.279 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:33:28.279 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2023-12-01T06:33:28.279 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-dump/3 2023-12-01T06:33:28.279 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:33:28.279 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=f34c1808-1341-49c0-a404-7ac148cae925 2023-12-01T06:33:28.280 INFO:tasks.workunit.client.0.smithi170.stdout:add osd3 f34c1808-1341-49c0-a404-7ac148cae925 2023-12-01T06:33:28.280 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 f34c1808-1341-49c0-a404-7ac148cae925' 2023-12-01T06:33:28.280 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:33:28.303 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA4fmllXApOEhAAoZv0wzBmeQDQWlq6QIeGDA== 2023-12-01T06:33:28.303 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA4fmllXApOEhAAoZv0wzBmeQDQWlq6QIeGDA=="}' 2023-12-01T06:33:28.303 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new f34c1808-1341-49c0-a404-7ac148cae925 -i td/osd-scrub-dump/3/new.json 2023-12-01T06:33:28.667 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:33:28.685 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-dump/3/new.json 2023-12-01T06:33:28.686 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-dump/3 --osd-journal=td/osd-scrub-dump/3/journal --chdir= --run-dir=td/osd-scrub-dump '--admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-dump/$name.log' '--pid-file=td/osd-scrub-dump/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA4fmllXApOEhAAoZv0wzBmeQDQWlq6QIeGDA== --osd-uuid f34c1808-1341-49c0-a404-7ac148cae925 2023-12-01T06:33:28.729 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:28.731+0000 7f491abf9640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:28.751 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:28.751+0000 7f491abf9640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:28.752 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:28.755+0000 7f491abf9640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:28.752 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:28.755+0000 7f491abf9640 -1 bluestore(td/osd-scrub-dump/3/block) _read_bdev_label failed to open td/osd-scrub-dump/3/block: (2) No such file or directory 2023-12-01T06:33:28.753 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:28.755+0000 7f491abf9640 -1 bluestore(td/osd-scrub-dump/3/block) _read_bdev_label failed to open td/osd-scrub-dump/3/block: (2) No such file or directory 2023-12-01T06:33:28.753 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:28.755+0000 7f491abf9640 -1 bluestore(td/osd-scrub-dump/3/block) _read_bdev_label failed to open td/osd-scrub-dump/3/block: (2) No such file or directory 2023-12-01T06:33:28.859 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:28.863+0000 7f491abf9640 -1 bluestore(td/osd-scrub-dump/3) _read_fsid unparsable uuid 2023-12-01T06:33:35.198 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-dump/3/keyring 2023-12-01T06:33:35.198 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:33:35.199 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd3 key to auth repository 2023-12-01T06:33:35.199 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2023-12-01T06:33:35.199 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-dump/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:33:35.590 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2023-12-01T06:33:35.591 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.3 2023-12-01T06:33:35.591 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-dump/3 --osd-journal=td/osd-scrub-dump/3/journal --chdir= --run-dir=td/osd-scrub-dump '--admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-dump/$name.log' '--pid-file=td/osd-scrub-dump/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2023-12-01T06:33:35.591 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:33:35.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:33:35.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:33:35.631 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:35.631+0000 7f510be36640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:35.659 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:35.659+0000 7f510be36640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:35.660 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:35.663+0000 7f510be36640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:35.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2023-12-01T06:33:35.924 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:33:35.924 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2023-12-01T06:33:35.924 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:33:35.924 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:33:35.924 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:35.924 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:33:35.924 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:33:35.924 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:35.925 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2023-12-01T06:33:36.202 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:36.203+0000 7f510be36640 -1 Falling back to public interface 2023-12-01T06:33:36.240 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:37.241 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:37.241 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:37.242 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:33:37.242 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:33:37.242 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:37.242 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2023-12-01T06:33:37.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:38.550 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:38.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:38.551 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:33:38.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:33:38.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:38.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2023-12-01T06:33:38.721 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:38.723+0000 7f510be36640 -1 osd.3 0 log_to_monitors true 2023-12-01T06:33:38.944 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:39.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:39.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:39.946 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:33:39.947 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:33:39.947 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:39.947 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2023-12-01T06:33:40.289 INFO:tasks.workunit.client.0.smithi170.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1944303157,v1:127.0.0.1:6827/1944303157] [v2:127.0.0.1:6828/1944303157,v1:127.0.0.1:6829/1944303157] exists,up f34c1808-1341-49c0-a404-7ac148cae925 2023-12-01T06:33:40.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:33:40.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:33:40.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:33:40.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:61: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:33:40.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:63: TEST_recover_unexpected: run_osd td/osd-scrub-dump 4 2023-12-01T06:33:40.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-dump 2023-12-01T06:33:40.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:33:40.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=4 2023-12-01T06:33:40.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:33:40.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-dump/4 2023-12-01T06:33:40.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq ' 2023-12-01T06:33:40.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:33:40.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:33:40.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:33:40.292 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-dump/4' 2023-12-01T06:33:40.292 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-dump/4/journal' 2023-12-01T06:33:40.292 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:33:40.292 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:33:40.292 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-dump' 2023-12-01T06:33:40.292 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:33:40.292 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:33:40.292 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:33:40.293 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:33:40.293 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:33:40.293 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:33:40.293 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.72213/$cluster-$name.asok' 2023-12-01T06:33:40.293 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' 2023-12-01T06:33:40.293 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:33:40.293 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:33:40.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:33:40.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-dump/$name.log' 2023-12-01T06:33:40.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-dump/$name.pid' 2023-12-01T06:33:40.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:33:40.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:33:40.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:33:40.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:33:40.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:33:40.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2023-12-01T06:33:40.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-dump/4 2023-12-01T06:33:40.295 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:33:40.295 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=51e51d52-f208-48ce-bd55-346e96b8d5f4 2023-12-01T06:33:40.296 INFO:tasks.workunit.client.0.smithi170.stdout:add osd4 51e51d52-f208-48ce-bd55-346e96b8d5f4 2023-12-01T06:33:40.296 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd4 51e51d52-f208-48ce-bd55-346e96b8d5f4' 2023-12-01T06:33:40.296 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:33:40.324 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBEfmllWJeSExAAt64YAQIRyg0Pa6txiX/mlA== 2023-12-01T06:33:40.325 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBEfmllWJeSExAAt64YAQIRyg0Pa6txiX/mlA=="}' 2023-12-01T06:33:40.325 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 51e51d52-f208-48ce-bd55-346e96b8d5f4 -i td/osd-scrub-dump/4/new.json 2023-12-01T06:33:40.674 INFO:tasks.workunit.client.0.smithi170.stdout:4 2023-12-01T06:33:40.692 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-dump/4/new.json 2023-12-01T06:33:40.693 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 4 --fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-dump/4 --osd-journal=td/osd-scrub-dump/4/journal --chdir= --run-dir=td/osd-scrub-dump '--admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-dump/$name.log' '--pid-file=td/osd-scrub-dump/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBEfmllWJeSExAAt64YAQIRyg0Pa6txiX/mlA== --osd-uuid 51e51d52-f208-48ce-bd55-346e96b8d5f4 2023-12-01T06:33:40.736 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:40.739+0000 7f5acf70f640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:40.763 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:40.763+0000 7f5acf70f640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:40.764 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:40.767+0000 7f5acf70f640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:40.764 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:40.767+0000 7f5acf70f640 -1 bluestore(td/osd-scrub-dump/4/block) _read_bdev_label failed to open td/osd-scrub-dump/4/block: (2) No such file or directory 2023-12-01T06:33:40.765 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:40.767+0000 7f5acf70f640 -1 bluestore(td/osd-scrub-dump/4/block) _read_bdev_label failed to open td/osd-scrub-dump/4/block: (2) No such file or directory 2023-12-01T06:33:40.765 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:40.767+0000 7f5acf70f640 -1 bluestore(td/osd-scrub-dump/4/block) _read_bdev_label failed to open td/osd-scrub-dump/4/block: (2) No such file or directory 2023-12-01T06:33:40.842 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:40.843+0000 7f5acf70f640 -1 bluestore(td/osd-scrub-dump/4) _read_fsid unparsable uuid 2023-12-01T06:33:47.556 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-dump/4/keyring 2023-12-01T06:33:47.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:33:47.557 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd4 key to auth repository 2023-12-01T06:33:47.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd4 key to auth repository 2023-12-01T06:33:47.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-dump/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:33:47.951 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.4 2023-12-01T06:33:47.952 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.4 2023-12-01T06:33:47.952 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:33:47.952 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 4 --fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-dump/4 --osd-journal=td/osd-scrub-dump/4/journal --chdir= --run-dir=td/osd-scrub-dump '--admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-dump/$name.log' '--pid-file=td/osd-scrub-dump/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2023-12-01T06:33:47.952 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:33:47.952 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:33:47.992 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:47.995+0000 7f88e1f48640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:48.019 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:48.023+0000 7f88e1f48640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:48.021 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:48.023+0000 7f88e1f48640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:48.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 4 2023-12-01T06:33:48.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:33:48.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=4 2023-12-01T06:33:48.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:33:48.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:33:48.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:48.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:33:48.284 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:33:48.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:48.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2023-12-01T06:33:48.570 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:48.571+0000 7f88e1f48640 -1 Falling back to public interface 2023-12-01T06:33:48.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:49.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:49.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:49.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:33:49.620 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:33:49.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:49.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2023-12-01T06:33:49.942 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:50.944 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:50.944 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:50.944 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:33:50.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:33:50.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:50.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2023-12-01T06:33:51.123 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:51.123+0000 7f88e1f48640 -1 osd.4 0 log_to_monitors true 2023-12-01T06:33:51.272 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:33:52.274 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:33:52.274 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:33:52.274 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:33:52.274 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:33:52.275 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:33:52.275 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2023-12-01T06:33:52.633 INFO:tasks.workunit.client.0.smithi170.stdout:osd.4 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/208551569,v1:127.0.0.1:6835/208551569] [v2:127.0.0.1:6836/208551569,v1:127.0.0.1:6837/208551569] exists,up 51e51d52-f208-48ce-bd55-346e96b8d5f4 2023-12-01T06:33:52.634 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:33:52.634 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:61: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:63: TEST_recover_unexpected: run_osd td/osd-scrub-dump 5 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-dump 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=5 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-dump/5 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq ' 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-dump/5' 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-dump/5/journal' 2023-12-01T06:33:52.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-12-01T06:33:52.636 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-12-01T06:33:52.636 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-dump' 2023-12-01T06:33:52.636 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-12-01T06:33:52.636 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-12-01T06:33:52.636 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-12-01T06:33:52.636 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-12-01T06:33:52.636 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:33:52.636 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:33:52.636 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.72213/$cluster-$name.asok' 2023-12-01T06:33:52.637 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' 2023-12-01T06:33:52.637 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-12-01T06:33:52.637 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-12-01T06:33:52.637 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-12-01T06:33:52.637 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-dump/$name.log' 2023-12-01T06:33:52.637 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-dump/$name.pid' 2023-12-01T06:33:52.637 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-12-01T06:33:52.637 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-12-01T06:33:52.637 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-12-01T06:33:52.638 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2023-12-01T06:33:52.638 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2023-12-01T06:33:52.638 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2023-12-01T06:33:52.638 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-dump/5 2023-12-01T06:33:52.638 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2023-12-01T06:33:52.639 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=46a6c723-b8e6-4272-8a61-053ca592cae6 2023-12-01T06:33:52.639 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd5 46a6c723-b8e6-4272-8a61-053ca592cae6' 2023-12-01T06:33:52.639 INFO:tasks.workunit.client.0.smithi170.stdout:add osd5 46a6c723-b8e6-4272-8a61-053ca592cae6 2023-12-01T06:33:52.639 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2023-12-01T06:33:52.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBQfmllo3L4JxAA/gs7eHwpZPSvqdjPDfvDdA== 2023-12-01T06:33:52.667 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBQfmllo3L4JxAA/gs7eHwpZPSvqdjPDfvDdA=="}' 2023-12-01T06:33:52.667 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 46a6c723-b8e6-4272-8a61-053ca592cae6 -i td/osd-scrub-dump/5/new.json 2023-12-01T06:33:53.020 INFO:tasks.workunit.client.0.smithi170.stdout:5 2023-12-01T06:33:53.037 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-dump/5/new.json 2023-12-01T06:33:53.038 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 5 --fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-dump/5 --osd-journal=td/osd-scrub-dump/5/journal --chdir= --run-dir=td/osd-scrub-dump '--admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-dump/$name.log' '--pid-file=td/osd-scrub-dump/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBQfmllo3L4JxAA/gs7eHwpZPSvqdjPDfvDdA== --osd-uuid 46a6c723-b8e6-4272-8a61-053ca592cae6 2023-12-01T06:33:53.074 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:53.075+0000 7f6612f58640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:53.104 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:53.107+0000 7f6612f58640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:53.106 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:53.107+0000 7f6612f58640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:33:53.106 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:53.107+0000 7f6612f58640 -1 bluestore(td/osd-scrub-dump/5/block) _read_bdev_label failed to open td/osd-scrub-dump/5/block: (2) No such file or directory 2023-12-01T06:33:53.107 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:53.107+0000 7f6612f58640 -1 bluestore(td/osd-scrub-dump/5/block) _read_bdev_label failed to open td/osd-scrub-dump/5/block: (2) No such file or directory 2023-12-01T06:33:53.107 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:53.107+0000 7f6612f58640 -1 bluestore(td/osd-scrub-dump/5/block) _read_bdev_label failed to open td/osd-scrub-dump/5/block: (2) No such file or directory 2023-12-01T06:33:53.206 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:33:53.207+0000 7f6612f58640 -1 bluestore(td/osd-scrub-dump/5) _read_fsid unparsable uuid 2023-12-01T06:33:59.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-dump/5/keyring 2023-12-01T06:33:59.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2023-12-01T06:33:59.859 INFO:tasks.workunit.client.0.smithi170.stdout:adding osd5 key to auth repository 2023-12-01T06:33:59.859 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd5 key to auth repository 2023-12-01T06:33:59.859 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-dump/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-12-01T06:34:00.252 INFO:tasks.workunit.client.0.smithi170.stdout:start osd.5 2023-12-01T06:34:00.252 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.5 2023-12-01T06:34:00.253 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 5 --fsid=21d5adad-81fe-4f4f-8a8c-f16ee85bd4a6 --auth-supported=none --mon-host=127.0.0.1:7184 --osd_max_scrubs=4 --osd_shallow_scrub_chunk_max=5 --osd_scrub_sleep=3 --osd_pool_default_size=3 --osd_op_queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-dump/5 --osd-journal=td/osd-scrub-dump/5/journal --chdir= --run-dir=td/osd-scrub-dump '--admin-socket=/tmp/ceph-asok.72213/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-dump/$name.log' '--pid-file=td/osd-scrub-dump/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2023-12-01T06:34:00.253 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2023-12-01T06:34:00.253 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2023-12-01T06:34:00.253 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2023-12-01T06:34:00.294 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:34:00.295+0000 7f60e8edc640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:34:00.315 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:34:00.315+0000 7f60e8edc640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:34:00.316 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:34:00.319+0000 7f60e8edc640 -1 WARNING: all dangerous and experimental features are enabled. 2023-12-01T06:34:00.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 5 2023-12-01T06:34:00.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2023-12-01T06:34:00.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2023-12-01T06:34:00.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2023-12-01T06:34:00.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2023-12-01T06:34:00.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:34:00.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2023-12-01T06:34:00.577 INFO:tasks.workunit.client.0.smithi170.stdout:0 2023-12-01T06:34:00.577 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:34:00.577 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2023-12-01T06:34:00.861 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:34:00.863+0000 7f60e8edc640 -1 Falling back to public interface 2023-12-01T06:34:00.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:34:01.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:34:01.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:34:01.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2023-12-01T06:34:01.891 INFO:tasks.workunit.client.0.smithi170.stdout:1 2023-12-01T06:34:01.891 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:34:01.891 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2023-12-01T06:34:02.207 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:34:03.209 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:34:03.209 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:34:03.209 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2023-12-01T06:34:03.209 INFO:tasks.workunit.client.0.smithi170.stdout:2 2023-12-01T06:34:03.210 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:34:03.210 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2023-12-01T06:34:03.481 INFO:tasks.workunit.client.0.smithi170.stderr:2023-12-01T06:34:03.483+0000 7f60e8edc640 -1 osd.5 0 log_to_monitors true 2023-12-01T06:34:03.536 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2023-12-01T06:34:04.538 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2023-12-01T06:34:04.538 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2023-12-01T06:34:04.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2023-12-01T06:34:04.539 INFO:tasks.workunit.client.0.smithi170.stdout:3 2023-12-01T06:34:04.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2023-12-01T06:34:04.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2023-12-01T06:34:04.901 INFO:tasks.workunit.client.0.smithi170.stdout:osd.5 up in weight 1 up_from 30 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1790200020,v1:127.0.0.1:6843/1790200020] [v2:127.0.0.1:6844/1790200020,v1:127.0.0.1:6845/1790200020] exists,up 46a6c723-b8e6-4272-8a61-053ca592cae6 2023-12-01T06:34:04.901 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2023-12-01T06:34:04.902 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2023-12-01T06:34:04.902 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2023-12-01T06:34:04.902 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:66: TEST_recover_unexpected: seq 1 3 2023-12-01T06:34:04.904 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:66: TEST_recover_unexpected: for i in $(seq 1 $POOLS) 2023-12-01T06:34:04.904 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:68: TEST_recover_unexpected: create_pool test1 16 16 2023-12-01T06:34:04.904 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test1 16 16 2023-12-01T06:34:05.408 INFO:tasks.workunit.client.0.smithi170.stderr:pool 'test1' created 2023-12-01T06:34:05.570 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2023-12-01T06:34:06.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:66: TEST_recover_unexpected: for i in $(seq 1 $POOLS) 2023-12-01T06:34:06.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:68: TEST_recover_unexpected: create_pool test2 16 16 2023-12-01T06:34:06.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test2 16 16 2023-12-01T06:34:07.372 INFO:tasks.workunit.client.0.smithi170.stderr:pool 'test2' created 2023-12-01T06:34:07.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2023-12-01T06:34:08.397 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:66: TEST_recover_unexpected: for i in $(seq 1 $POOLS) 2023-12-01T06:34:08.398 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:68: TEST_recover_unexpected: create_pool test3 16 16 2023-12-01T06:34:08.398 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test3 16 16 2023-12-01T06:34:09.292 INFO:tasks.workunit.client.0.smithi170.stderr:pool 'test3' created 2023-12-01T06:34:09.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2023-12-01T06:34:10.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:71: TEST_recover_unexpected: wait_for_clean 2023-12-01T06:34:10.318 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1642: wait_for_clean: local cmd= 2023-12-01T06:34:10.318 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1643: wait_for_clean: local num_active_clean=-1 2023-12-01T06:34:10.318 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1644: wait_for_clean: local cur_active_clean 2023-12-01T06:34:10.319 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1645: wait_for_clean: get_timeout_delays 90 .1 2023-12-01T06:34:10.319 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1588: get_timeout_delays: shopt -q -o xtrace 2023-12-01T06:34:10.319 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1588: get_timeout_delays: echo true 2023-12-01T06:34:10.320 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1588: get_timeout_delays: local trace=true 2023-12-01T06:34:10.320 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1589: get_timeout_delays: true 2023-12-01T06:34:10.320 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1589: get_timeout_delays: shopt -u -o xtrace 2023-12-01T06:34:10.437 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1645: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2023-12-01T06:34:10.437 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1645: wait_for_clean: local -a delays 2023-12-01T06:34:10.438 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1646: wait_for_clean: local -i loop=0 2023-12-01T06:34:10.438 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: flush_pg_stats 2023-12-01T06:34:10.438 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2197: flush_pg_stats: local timeout=300 2023-12-01T06:34:10.438 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2199: flush_pg_stats: ceph osd ls 2023-12-01T06:34:10.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2199: flush_pg_stats: ids='0 2023-12-01T06:34:10.891 INFO:tasks.workunit.client.0.smithi170.stderr:1 2023-12-01T06:34:10.891 INFO:tasks.workunit.client.0.smithi170.stderr:2 2023-12-01T06:34:10.891 INFO:tasks.workunit.client.0.smithi170.stderr:3 2023-12-01T06:34:10.891 INFO:tasks.workunit.client.0.smithi170.stderr:4 2023-12-01T06:34:10.891 INFO:tasks.workunit.client.0.smithi170.stderr:5' 2023-12-01T06:34:10.891 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2200: flush_pg_stats: seqs= 2023-12-01T06:34:10.891 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:34:10.891 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2023-12-01T06:34:11.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=21474836495 2023-12-01T06:34:11.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 21474836495 2023-12-01T06:34:11.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836495' 2023-12-01T06:34:11.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:34:11.147 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2023-12-01T06:34:11.321 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=42949672973 2023-12-01T06:34:11.321 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 42949672973 2023-12-01T06:34:11.321 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836495 1-42949672973' 2023-12-01T06:34:11.321 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:34:11.321 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2023-12-01T06:34:11.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=64424509450 2023-12-01T06:34:11.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 64424509450 2023-12-01T06:34:11.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836495 1-42949672973 2-64424509450' 2023-12-01T06:34:11.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:34:11.475 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2023-12-01T06:34:11.633 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=85899345929 2023-12-01T06:34:11.634 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 85899345929 2023-12-01T06:34:11.634 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836495 1-42949672973 2-64424509450 3-85899345929' 2023-12-01T06:34:11.634 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:34:11.634 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2023-12-01T06:34:11.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=107374182405 2023-12-01T06:34:11.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 107374182405 2023-12-01T06:34:11.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836495 1-42949672973 2-64424509450 3-85899345929 4-107374182405' 2023-12-01T06:34:11.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2201: flush_pg_stats: for osd in $ids 2023-12-01T06:34:11.793 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2023-12-01T06:34:11.949 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2202: flush_pg_stats: seq=128849018884 2023-12-01T06:34:11.949 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2203: flush_pg_stats: test -z 128849018884 2023-12-01T06:34:11.949 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2207: flush_pg_stats: seqs=' 0-21474836495 1-42949672973 2-64424509450 3-85899345929 4-107374182405 5-128849018884' 2023-12-01T06:34:11.949 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:34:11.950 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 0-21474836495 2023-12-01T06:34:11.950 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:34:11.951 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=0 2023-12-01T06:34:11.952 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 0-21474836495 2023-12-01T06:34:11.952 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:34:11.954 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=21474836495 2023-12-01T06:34:11.955 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.0 seq 21474836495 2023-12-01T06:34:11.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2023-12-01T06:34:11.955 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 0 2023-12-01T06:34:12.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 21474836494 -lt 21474836495 2023-12-01T06:34:12.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2215: flush_pg_stats: sleep 1 2023-12-01T06:34:13.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2216: flush_pg_stats: '[' 300 -eq 0 ']' 2023-12-01T06:34:13.277 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 0 2023-12-01T06:34:13.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 21474836495 -lt 21474836495 2023-12-01T06:34:13.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:34:13.604 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 1-42949672973 2023-12-01T06:34:13.604 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:34:13.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=1 2023-12-01T06:34:13.606 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 1-42949672973 2023-12-01T06:34:13.606 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:34:13.608 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=42949672973 2023-12-01T06:34:13.608 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.1 seq 42949672973 2023-12-01T06:34:13.608 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.1 seq 42949672973' 2023-12-01T06:34:13.609 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 1 2023-12-01T06:34:13.914 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 42949672973 -lt 42949672973 2023-12-01T06:34:13.914 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:34:13.915 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 2-64424509450 2023-12-01T06:34:13.915 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:34:13.916 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=2 2023-12-01T06:34:13.917 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 2-64424509450 2023-12-01T06:34:13.918 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:34:13.919 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=64424509450 2023-12-01T06:34:13.919 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.2 seq 64424509450' 2023-12-01T06:34:13.919 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.2 seq 64424509450 2023-12-01T06:34:13.919 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 2 2023-12-01T06:34:14.234 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 64424509450 -lt 64424509450 2023-12-01T06:34:14.234 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:34:14.239 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 3-85899345929 2023-12-01T06:34:14.239 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:34:14.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=3 2023-12-01T06:34:14.239 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 3-85899345929 2023-12-01T06:34:14.239 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:34:14.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=85899345929 2023-12-01T06:34:14.239 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.3 seq 85899345929 2023-12-01T06:34:14.240 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.3 seq 85899345929' 2023-12-01T06:34:14.240 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 3 2023-12-01T06:34:14.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 85899345929 -lt 85899345929 2023-12-01T06:34:14.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:34:14.547 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 4-107374182405 2023-12-01T06:34:14.548 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:34:14.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=4 2023-12-01T06:34:14.550 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 4-107374182405 2023-12-01T06:34:14.550 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:34:14.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=107374182405 2023-12-01T06:34:14.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.4 seq 107374182405' 2023-12-01T06:34:14.552 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.4 seq 107374182405 2023-12-01T06:34:14.552 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 4 2023-12-01T06:34:14.870 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 107374182406 -lt 107374182405 2023-12-01T06:34:14.870 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2210: flush_pg_stats: for s in $seqs 2023-12-01T06:34:14.870 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: echo 5-128849018884 2023-12-01T06:34:14.871 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: cut -d - -f 1 2023-12-01T06:34:14.872 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2211: flush_pg_stats: osd=5 2023-12-01T06:34:14.873 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: echo 5-128849018884 2023-12-01T06:34:14.873 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: cut -d - -f 2 2023-12-01T06:34:14.873 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2212: flush_pg_stats: seq=128849018884 2023-12-01T06:34:14.874 INFO:tasks.workunit.client.0.smithi170.stdout:waiting osd.5 seq 128849018884 2023-12-01T06:34:14.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2213: flush_pg_stats: echo 'waiting osd.5 seq 128849018884' 2023-12-01T06:34:14.874 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: ceph osd last-stat-seq 5 2023-12-01T06:34:15.196 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2214: flush_pg_stats: test 128849018884 -lt 128849018884 2023-12-01T06:34:15.197 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: get_num_pgs 2023-12-01T06:34:15.197 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2023-12-01T06:34:15.197 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2023-12-01T06:34:15.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: test 48 == 0 2023-12-01T06:34:15.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: true 2023-12-01T06:34:15.582 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: get_num_active_clean 2023-12-01T06:34:15.582 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2023-12-01T06:34:15.583 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2023-12-01T06:34:15.583 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2023-12-01T06:34:15.583 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2023-12-01T06:34:15.583 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2023-12-01T06:34:15.933 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: cur_active_clean=48 2023-12-01T06:34:15.934 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: get_num_pgs 2023-12-01T06:34:15.934 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2023-12-01T06:34:15.934 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2023-12-01T06:34:16.321 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: test 48 = 48 2023-12-01T06:34:16.321 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: break 2023-12-01T06:34:16.321 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: return 0 2023-12-01T06:34:16.321 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:73: TEST_recover_unexpected: dd if=/dev/urandom of=datafile bs=4k count=2 2023-12-01T06:34:16.322 INFO:tasks.workunit.client.0.smithi170.stderr:2+0 records in 2023-12-01T06:34:16.322 INFO:tasks.workunit.client.0.smithi170.stderr:2+0 records out 2023-12-01T06:34:16.322 INFO:tasks.workunit.client.0.smithi170.stderr:8192 bytes (8.2 kB, 8.0 KiB) copied, 0.000274006 s, 29.9 MB/s 2023-12-01T06:34:16.323 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:74: TEST_recover_unexpected: seq 1 3 2023-12-01T06:34:16.324 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:74: TEST_recover_unexpected: for i in $(seq 1 $POOLS) 2023-12-01T06:34:16.325 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: seq 1 1000 2023-12-01T06:34:16.325 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:16.325 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj1 datafile 2023-12-01T06:34:16.504 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:16.505 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj2 datafile 2023-12-01T06:34:16.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:16.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj3 datafile 2023-12-01T06:34:16.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:16.840 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj4 datafile 2023-12-01T06:34:17.016 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:17.017 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj5 datafile 2023-12-01T06:34:17.185 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:17.185 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj6 datafile 2023-12-01T06:34:17.352 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:17.352 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj7 datafile 2023-12-01T06:34:17.511 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:17.511 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj8 datafile 2023-12-01T06:34:17.669 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:17.669 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj9 datafile 2023-12-01T06:34:17.846 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:17.846 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj10 datafile 2023-12-01T06:34:17.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:17.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj11 datafile 2023-12-01T06:34:18.166 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:18.166 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj12 datafile 2023-12-01T06:34:18.333 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:18.333 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj13 datafile 2023-12-01T06:34:18.524 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:18.524 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj14 datafile 2023-12-01T06:34:18.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:18.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj15 datafile 2023-12-01T06:34:18.853 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:18.853 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj16 datafile 2023-12-01T06:34:19.002 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:19.003 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj17 datafile 2023-12-01T06:34:19.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:19.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj18 datafile 2023-12-01T06:34:19.330 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:19.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj19 datafile 2023-12-01T06:34:19.489 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:19.489 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj20 datafile 2023-12-01T06:34:19.656 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:19.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj21 datafile 2023-12-01T06:34:19.834 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:19.834 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj22 datafile 2023-12-01T06:34:19.992 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:19.992 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj23 datafile 2023-12-01T06:34:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj24 datafile 2023-12-01T06:34:20.302 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:20.302 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj25 datafile 2023-12-01T06:34:20.445 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:20.445 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj26 datafile 2023-12-01T06:34:20.613 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:20.613 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj27 datafile 2023-12-01T06:34:20.771 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:20.772 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj28 datafile 2023-12-01T06:34:20.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:20.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj29 datafile 2023-12-01T06:34:21.108 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:21.108 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj30 datafile 2023-12-01T06:34:21.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:21.284 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj31 datafile 2023-12-01T06:34:21.451 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:21.451 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj32 datafile 2023-12-01T06:34:21.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:21.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj33 datafile 2023-12-01T06:34:21.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:21.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj34 datafile 2023-12-01T06:34:21.995 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:21.996 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj35 datafile 2023-12-01T06:34:22.180 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:22.181 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj36 datafile 2023-12-01T06:34:22.330 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:22.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj37 datafile 2023-12-01T06:34:22.499 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:22.499 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj38 datafile 2023-12-01T06:34:22.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:22.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj39 datafile 2023-12-01T06:34:22.833 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:22.834 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj40 datafile 2023-12-01T06:34:23.003 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:23.003 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj41 datafile 2023-12-01T06:34:23.162 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:23.163 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj42 datafile 2023-12-01T06:34:23.320 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:23.320 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj43 datafile 2023-12-01T06:34:23.479 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:23.479 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj44 datafile 2023-12-01T06:34:23.630 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:23.631 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj45 datafile 2023-12-01T06:34:23.814 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:23.814 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj46 datafile 2023-12-01T06:34:24.001 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:24.002 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj47 datafile 2023-12-01T06:34:24.194 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:24.194 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj48 datafile 2023-12-01T06:34:24.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:24.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj49 datafile 2023-12-01T06:34:24.504 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:24.504 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj50 datafile 2023-12-01T06:34:24.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:24.655 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj51 datafile 2023-12-01T06:34:24.797 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:24.797 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj52 datafile 2023-12-01T06:34:24.949 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:24.949 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj53 datafile 2023-12-01T06:34:25.107 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:25.107 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj54 datafile 2023-12-01T06:34:25.266 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:25.266 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj55 datafile 2023-12-01T06:34:25.434 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:25.435 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj56 datafile 2023-12-01T06:34:25.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:25.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj57 datafile 2023-12-01T06:34:25.760 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:25.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj58 datafile 2023-12-01T06:34:25.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:25.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj59 datafile 2023-12-01T06:34:26.096 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:26.096 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj60 datafile 2023-12-01T06:34:26.247 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:26.247 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj61 datafile 2023-12-01T06:34:26.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:26.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj62 datafile 2023-12-01T06:34:26.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:26.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj63 datafile 2023-12-01T06:34:26.699 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:26.700 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj64 datafile 2023-12-01T06:34:26.884 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:26.884 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj65 datafile 2023-12-01T06:34:27.085 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:27.086 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj66 datafile 2023-12-01T06:34:27.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:27.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj67 datafile 2023-12-01T06:34:27.495 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:27.496 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj68 datafile 2023-12-01T06:34:27.646 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:27.646 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj69 datafile 2023-12-01T06:34:27.831 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:27.831 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj70 datafile 2023-12-01T06:34:27.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:27.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj71 datafile 2023-12-01T06:34:28.157 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:28.157 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj72 datafile 2023-12-01T06:34:28.308 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:28.308 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj73 datafile 2023-12-01T06:34:28.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:28.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj74 datafile 2023-12-01T06:34:28.643 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:28.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj75 datafile 2023-12-01T06:34:28.803 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:28.803 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj76 datafile 2023-12-01T06:34:28.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:28.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj77 datafile 2023-12-01T06:34:29.115 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:29.115 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj78 datafile 2023-12-01T06:34:29.289 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:29.289 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj79 datafile 2023-12-01T06:34:29.431 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:29.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj80 datafile 2023-12-01T06:34:29.590 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:29.590 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj81 datafile 2023-12-01T06:34:29.749 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:29.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj82 datafile 2023-12-01T06:34:29.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:29.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj83 datafile 2023-12-01T06:34:30.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:30.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj84 datafile 2023-12-01T06:34:30.328 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:30.328 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj85 datafile 2023-12-01T06:34:30.487 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:30.488 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj86 datafile 2023-12-01T06:34:30.672 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:30.672 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj87 datafile 2023-12-01T06:34:30.815 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:30.815 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj88 datafile 2023-12-01T06:34:30.965 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:30.965 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj89 datafile 2023-12-01T06:34:31.125 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:31.125 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj90 datafile 2023-12-01T06:34:31.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:31.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj91 datafile 2023-12-01T06:34:31.443 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:31.443 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj92 datafile 2023-12-01T06:34:31.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:31.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj93 datafile 2023-12-01T06:34:31.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:31.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj94 datafile 2023-12-01T06:34:31.938 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:31.938 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj95 datafile 2023-12-01T06:34:32.105 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:32.105 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj96 datafile 2023-12-01T06:34:32.257 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:32.257 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj97 datafile 2023-12-01T06:34:32.651 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:32.652 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj98 datafile 2023-12-01T06:34:32.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:32.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj99 datafile 2023-12-01T06:34:32.984 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:32.985 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj100 datafile 2023-12-01T06:34:33.136 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:33.137 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj101 datafile 2023-12-01T06:34:33.304 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:33.304 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj102 datafile 2023-12-01T06:34:33.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:33.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj103 datafile 2023-12-01T06:34:33.621 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:33.622 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj104 datafile 2023-12-01T06:34:33.797 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:33.797 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj105 datafile 2023-12-01T06:34:33.958 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:33.958 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj106 datafile 2023-12-01T06:34:34.116 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:34.116 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj107 datafile 2023-12-01T06:34:34.275 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:34.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj108 datafile 2023-12-01T06:34:34.427 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:34.427 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj109 datafile 2023-12-01T06:34:34.586 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:34.586 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj110 datafile 2023-12-01T06:34:34.745 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:34.745 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj111 datafile 2023-12-01T06:34:34.896 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:34.896 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj112 datafile 2023-12-01T06:34:35.047 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:35.047 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj113 datafile 2023-12-01T06:34:35.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:35.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj114 datafile 2023-12-01T06:34:35.365 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:35.365 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj115 datafile 2023-12-01T06:34:35.516 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:35.516 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj116 datafile 2023-12-01T06:34:35.675 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:35.675 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj117 datafile 2023-12-01T06:34:35.852 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:35.852 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj118 datafile 2023-12-01T06:34:36.002 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:36.031 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj119 datafile 2023-12-01T06:34:36.153 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:36.153 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj120 datafile 2023-12-01T06:34:36.304 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:36.304 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj121 datafile 2023-12-01T06:34:36.463 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:36.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj122 datafile 2023-12-01T06:34:36.623 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:36.623 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj123 datafile 2023-12-01T06:34:36.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:36.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj124 datafile 2023-12-01T06:34:36.941 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:36.941 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj125 datafile 2023-12-01T06:34:37.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:37.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj126 datafile 2023-12-01T06:34:37.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:37.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj127 datafile 2023-12-01T06:34:37.410 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:37.410 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj128 datafile 2023-12-01T06:34:37.553 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:37.553 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj129 datafile 2023-12-01T06:34:37.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:37.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj130 datafile 2023-12-01T06:34:38.056 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:38.056 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj131 datafile 2023-12-01T06:34:38.234 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:38.234 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj132 datafile 2023-12-01T06:34:38.390 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:38.390 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj133 datafile 2023-12-01T06:34:38.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:38.550 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj134 datafile 2023-12-01T06:34:38.709 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:38.709 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj135 datafile 2023-12-01T06:34:38.893 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:38.893 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj136 datafile 2023-12-01T06:34:39.044 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:39.045 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj137 datafile 2023-12-01T06:34:39.204 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:39.204 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj138 datafile 2023-12-01T06:34:39.371 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:39.371 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj139 datafile 2023-12-01T06:34:39.514 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:39.514 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj140 datafile 2023-12-01T06:34:39.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:39.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj141 datafile 2023-12-01T06:34:39.832 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:39.832 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj142 datafile 2023-12-01T06:34:39.975 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:39.975 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj143 datafile 2023-12-01T06:34:40.135 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:40.135 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj144 datafile 2023-12-01T06:34:40.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:40.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj145 datafile 2023-12-01T06:34:40.453 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:40.453 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj146 datafile 2023-12-01T06:34:40.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:40.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj147 datafile 2023-12-01T06:34:40.754 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:40.754 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj148 datafile 2023-12-01T06:34:40.905 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:40.906 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj149 datafile 2023-12-01T06:34:41.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:41.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj150 datafile 2023-12-01T06:34:41.258 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:41.258 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj151 datafile 2023-12-01T06:34:41.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:41.400 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj152 datafile 2023-12-01T06:34:41.567 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:41.567 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj153 datafile 2023-12-01T06:34:41.727 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:41.727 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj154 datafile 2023-12-01T06:34:41.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:41.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj155 datafile 2023-12-01T06:34:42.096 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:42.096 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj156 datafile 2023-12-01T06:34:42.256 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:42.256 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj157 datafile 2023-12-01T06:34:42.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:42.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj158 datafile 2023-12-01T06:34:42.574 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:42.574 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj159 datafile 2023-12-01T06:34:42.724 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:42.724 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj160 datafile 2023-12-01T06:34:42.925 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:42.925 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj161 datafile 2023-12-01T06:34:43.084 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:43.084 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj162 datafile 2023-12-01T06:34:43.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:43.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj163 datafile 2023-12-01T06:34:43.394 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:43.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj164 datafile 2023-12-01T06:34:43.554 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:43.554 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj165 datafile 2023-12-01T06:34:43.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:43.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj166 datafile 2023-12-01T06:34:43.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:43.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj167 datafile 2023-12-01T06:34:44.082 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:44.083 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj168 datafile 2023-12-01T06:34:44.242 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:44.242 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj169 datafile 2023-12-01T06:34:44.392 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:44.392 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj170 datafile 2023-12-01T06:34:44.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:44.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj171 datafile 2023-12-01T06:34:44.711 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:44.711 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj172 datafile 2023-12-01T06:34:44.901 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:44.901 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj173 datafile 2023-12-01T06:34:45.053 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:45.053 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj174 datafile 2023-12-01T06:34:45.218 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:45.218 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj175 datafile 2023-12-01T06:34:45.372 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:45.372 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj176 datafile 2023-12-01T06:34:45.524 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:45.524 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj177 datafile 2023-12-01T06:34:45.691 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:45.691 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj178 datafile 2023-12-01T06:34:45.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:45.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj179 datafile 2023-12-01T06:34:46.010 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:46.010 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj180 datafile 2023-12-01T06:34:46.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:46.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj181 datafile 2023-12-01T06:34:46.327 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:46.328 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj182 datafile 2023-12-01T06:34:46.487 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:46.488 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj183 datafile 2023-12-01T06:34:46.664 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:46.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj184 datafile 2023-12-01T06:34:46.831 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:46.832 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj185 datafile 2023-12-01T06:34:47.000 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:47.000 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj186 datafile 2023-12-01T06:34:47.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:47.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj187 datafile 2023-12-01T06:34:47.355 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:47.355 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj188 datafile 2023-12-01T06:34:47.531 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:47.531 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj189 datafile 2023-12-01T06:34:47.708 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:47.708 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj190 datafile 2023-12-01T06:34:47.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:47.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj191 datafile 2023-12-01T06:34:48.079 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:48.079 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj192 datafile 2023-12-01T06:34:48.238 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:48.238 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj193 datafile 2023-12-01T06:34:48.413 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:48.413 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj194 datafile 2023-12-01T06:34:48.567 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:48.568 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj195 datafile 2023-12-01T06:34:48.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:48.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj196 datafile 2023-12-01T06:34:48.869 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:48.869 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj197 datafile 2023-12-01T06:34:49.012 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:49.012 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj198 datafile 2023-12-01T06:34:49.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:49.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj199 datafile 2023-12-01T06:34:49.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:49.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj200 datafile 2023-12-01T06:34:49.492 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:49.492 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj201 datafile 2023-12-01T06:34:49.640 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:49.640 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj202 datafile 2023-12-01T06:34:49.802 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:49.803 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj203 datafile 2023-12-01T06:34:49.959 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:49.959 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj204 datafile 2023-12-01T06:34:50.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:50.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj205 datafile 2023-12-01T06:34:50.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:50.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj206 datafile 2023-12-01T06:34:50.445 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:50.445 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj207 datafile 2023-12-01T06:34:50.597 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:50.597 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj208 datafile 2023-12-01T06:34:50.755 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:50.755 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj209 datafile 2023-12-01T06:34:50.931 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:50.931 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj210 datafile 2023-12-01T06:34:51.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:51.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj211 datafile 2023-12-01T06:34:51.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:51.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj212 datafile 2023-12-01T06:34:51.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:51.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj213 datafile 2023-12-01T06:34:51.568 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:51.569 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj214 datafile 2023-12-01T06:34:51.727 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:51.727 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj215 datafile 2023-12-01T06:34:51.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:51.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj216 datafile 2023-12-01T06:34:52.047 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:52.047 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj217 datafile 2023-12-01T06:34:52.218 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:52.218 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj218 datafile 2023-12-01T06:34:52.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:52.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj219 datafile 2023-12-01T06:34:52.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:52.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj220 datafile 2023-12-01T06:34:52.708 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:52.708 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj221 datafile 2023-12-01T06:34:52.860 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:52.860 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj222 datafile 2023-12-01T06:34:53.010 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:53.010 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj223 datafile 2023-12-01T06:34:53.195 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:53.195 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj224 datafile 2023-12-01T06:34:53.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:53.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj225 datafile 2023-12-01T06:34:53.488 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:53.489 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj226 datafile 2023-12-01T06:34:53.647 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:53.647 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj227 datafile 2023-12-01T06:34:53.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:53.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj228 datafile 2023-12-01T06:34:54.008 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:54.008 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj229 datafile 2023-12-01T06:34:54.167 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:54.167 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj230 datafile 2023-12-01T06:34:54.336 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:54.336 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj231 datafile 2023-12-01T06:34:54.494 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:54.494 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj232 datafile 2023-12-01T06:34:54.661 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:54.661 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj233 datafile 2023-12-01T06:34:54.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:54.812 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj234 datafile 2023-12-01T06:34:54.964 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:54.964 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj235 datafile 2023-12-01T06:34:55.168 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:55.168 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj236 datafile 2023-12-01T06:34:55.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:55.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj237 datafile 2023-12-01T06:34:55.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:55.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj238 datafile 2023-12-01T06:34:55.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:55.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj239 datafile 2023-12-01T06:34:55.785 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:55.785 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj240 datafile 2023-12-01T06:34:55.935 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:55.936 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj241 datafile 2023-12-01T06:34:56.079 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:56.079 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj242 datafile 2023-12-01T06:34:56.229 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:56.229 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj243 datafile 2023-12-01T06:34:56.364 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:56.364 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj244 datafile 2023-12-01T06:34:56.514 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:56.515 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj245 datafile 2023-12-01T06:34:56.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:56.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj246 datafile 2023-12-01T06:34:56.808 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:56.808 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj247 datafile 2023-12-01T06:34:56.967 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:56.967 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj248 datafile 2023-12-01T06:34:57.127 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:57.127 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj249 datafile 2023-12-01T06:34:57.302 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:57.302 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj250 datafile 2023-12-01T06:34:57.445 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:57.445 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj251 datafile 2023-12-01T06:34:57.587 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:57.587 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj252 datafile 2023-12-01T06:34:57.755 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:57.756 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj253 datafile 2023-12-01T06:34:57.941 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:57.941 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj254 datafile 2023-12-01T06:34:58.082 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:58.083 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj255 datafile 2023-12-01T06:34:58.325 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:58.325 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj256 datafile 2023-12-01T06:34:58.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:58.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj257 datafile 2023-12-01T06:34:58.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:58.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj258 datafile 2023-12-01T06:34:58.770 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:58.770 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj259 datafile 2023-12-01T06:34:58.921 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:58.921 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj260 datafile 2023-12-01T06:34:59.064 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:59.064 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj261 datafile 2023-12-01T06:34:59.232 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:59.232 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj262 datafile 2023-12-01T06:34:59.374 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:59.374 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj263 datafile 2023-12-01T06:34:59.524 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:59.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj264 datafile 2023-12-01T06:34:59.667 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:59.667 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj265 datafile 2023-12-01T06:34:59.860 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:34:59.860 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj266 datafile 2023-12-01T06:35:00.044 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:00.045 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj267 datafile 2023-12-01T06:35:00.195 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:00.195 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj268 datafile 2023-12-01T06:35:00.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:00.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj269 datafile 2023-12-01T06:35:00.498 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:00.498 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj270 datafile 2023-12-01T06:35:00.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:00.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj271 datafile 2023-12-01T06:35:00.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:00.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj272 datafile 2023-12-01T06:35:00.999 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:01.000 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj273 datafile 2023-12-01T06:35:01.167 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:01.167 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj274 datafile 2023-12-01T06:35:01.310 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:01.310 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj275 datafile 2023-12-01T06:35:01.461 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:01.461 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj276 datafile 2023-12-01T06:35:01.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:01.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj277 datafile 2023-12-01T06:35:01.771 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:01.771 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj278 datafile 2023-12-01T06:35:01.940 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:01.940 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj279 datafile 2023-12-01T06:35:02.098 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:02.098 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj280 datafile 2023-12-01T06:35:02.257 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:02.258 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj281 datafile 2023-12-01T06:35:02.433 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:02.433 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj282 datafile 2023-12-01T06:35:02.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:02.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj283 datafile 2023-12-01T06:35:02.760 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:02.760 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj284 datafile 2023-12-01T06:35:02.944 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:02.944 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj285 datafile 2023-12-01T06:35:03.113 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:03.113 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj286 datafile 2023-12-01T06:35:03.271 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:03.272 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj287 datafile 2023-12-01T06:35:03.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:03.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj288 datafile 2023-12-01T06:35:03.764 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:03.765 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj289 datafile 2023-12-01T06:35:03.942 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:03.942 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj290 datafile 2023-12-01T06:35:04.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:04.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj291 datafile 2023-12-01T06:35:04.260 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:04.260 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj292 datafile 2023-12-01T06:35:04.427 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:04.427 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj293 datafile 2023-12-01T06:35:04.569 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:04.570 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj294 datafile 2023-12-01T06:35:04.729 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:04.729 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj295 datafile 2023-12-01T06:35:04.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:04.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj296 datafile 2023-12-01T06:35:05.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:05.031 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj297 datafile 2023-12-01T06:35:05.189 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:05.190 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj298 datafile 2023-12-01T06:35:05.349 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:05.349 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj299 datafile 2023-12-01T06:35:05.516 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:05.516 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj300 datafile 2023-12-01T06:35:05.676 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:05.676 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj301 datafile 2023-12-01T06:35:05.819 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:05.819 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj302 datafile 2023-12-01T06:35:05.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:05.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj303 datafile 2023-12-01T06:35:06.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:06.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj304 datafile 2023-12-01T06:35:06.280 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:06.280 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj305 datafile 2023-12-01T06:35:06.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:06.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj306 datafile 2023-12-01T06:35:06.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:06.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj307 datafile 2023-12-01T06:35:06.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:06.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj308 datafile 2023-12-01T06:35:06.908 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:06.909 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj309 datafile 2023-12-01T06:35:07.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:07.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj310 datafile 2023-12-01T06:35:07.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:07.245 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj311 datafile 2023-12-01T06:35:07.420 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:07.420 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj312 datafile 2023-12-01T06:35:07.579 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:07.579 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj313 datafile 2023-12-01T06:35:07.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:07.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj314 datafile 2023-12-01T06:35:07.898 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:07.898 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj315 datafile 2023-12-01T06:35:08.049 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:08.050 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj316 datafile 2023-12-01T06:35:08.200 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:08.200 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj317 datafile 2023-12-01T06:35:08.359 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:08.360 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj318 datafile 2023-12-01T06:35:08.511 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:08.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj319 datafile 2023-12-01T06:35:08.721 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:08.721 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj320 datafile 2023-12-01T06:35:08.973 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:08.973 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj321 datafile 2023-12-01T06:35:09.115 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:09.115 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj322 datafile 2023-12-01T06:35:09.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:09.308 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj323 datafile 2023-12-01T06:35:09.466 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:09.466 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj324 datafile 2023-12-01T06:35:09.609 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:09.609 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj325 datafile 2023-12-01T06:35:09.765 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:09.765 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj326 datafile 2023-12-01T06:35:09.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:09.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj327 datafile 2023-12-01T06:35:10.088 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:10.088 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj328 datafile 2023-12-01T06:35:10.255 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:10.255 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj329 datafile 2023-12-01T06:35:10.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:10.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj330 datafile 2023-12-01T06:35:10.565 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:10.565 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj331 datafile 2023-12-01T06:35:10.707 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:10.708 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj332 datafile 2023-12-01T06:35:10.883 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:10.884 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj333 datafile 2023-12-01T06:35:11.051 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:11.051 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj334 datafile 2023-12-01T06:35:11.232 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:11.232 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj335 datafile 2023-12-01T06:35:11.378 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:11.378 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj336 datafile 2023-12-01T06:35:11.554 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:11.554 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj337 datafile 2023-12-01T06:35:11.713 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:11.713 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj338 datafile 2023-12-01T06:35:11.917 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:11.917 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj339 datafile 2023-12-01T06:35:12.108 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:12.108 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj340 datafile 2023-12-01T06:35:12.267 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:12.267 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj341 datafile 2023-12-01T06:35:12.423 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:12.423 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj342 datafile 2023-12-01T06:35:12.569 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:12.569 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj343 datafile 2023-12-01T06:35:12.746 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:12.746 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj344 datafile 2023-12-01T06:35:12.905 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:12.905 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj345 datafile 2023-12-01T06:35:13.047 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:13.047 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj346 datafile 2023-12-01T06:35:13.207 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:13.207 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj347 datafile 2023-12-01T06:35:13.366 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:13.366 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj348 datafile 2023-12-01T06:35:13.534 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:13.534 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj349 datafile 2023-12-01T06:35:13.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:13.685 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj350 datafile 2023-12-01T06:35:13.861 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:13.861 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj351 datafile 2023-12-01T06:35:14.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:14.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj352 datafile 2023-12-01T06:35:14.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:14.180 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj353 datafile 2023-12-01T06:35:14.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:14.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj354 datafile 2023-12-01T06:35:14.489 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:14.490 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj355 datafile 2023-12-01T06:35:14.623 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:14.624 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj356 datafile 2023-12-01T06:35:14.791 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:14.791 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj357 datafile 2023-12-01T06:35:14.976 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:14.976 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj358 datafile 2023-12-01T06:35:15.135 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:15.136 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj359 datafile 2023-12-01T06:35:15.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:15.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj360 datafile 2023-12-01T06:35:15.453 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:15.454 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj361 datafile 2023-12-01T06:35:15.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:15.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj362 datafile 2023-12-01T06:35:15.773 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:15.773 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj363 datafile 2023-12-01T06:35:15.958 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:15.959 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj364 datafile 2023-12-01T06:35:16.108 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:16.108 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj365 datafile 2023-12-01T06:35:16.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:16.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj366 datafile 2023-12-01T06:35:16.418 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:16.418 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj367 datafile 2023-12-01T06:35:16.570 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:16.570 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj368 datafile 2023-12-01T06:35:16.729 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:16.729 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj369 datafile 2023-12-01T06:35:16.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:16.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj370 datafile 2023-12-01T06:35:17.040 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:17.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj371 datafile 2023-12-01T06:35:17.248 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:17.248 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj372 datafile 2023-12-01T06:35:17.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:17.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj373 datafile 2023-12-01T06:35:17.558 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:17.558 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj374 datafile 2023-12-01T06:35:17.735 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:17.735 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj375 datafile 2023-12-01T06:35:17.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:17.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj376 datafile 2023-12-01T06:35:18.112 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:18.112 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj377 datafile 2023-12-01T06:35:18.271 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:18.271 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj378 datafile 2023-12-01T06:35:18.422 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:18.422 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj379 datafile 2023-12-01T06:35:18.582 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:18.582 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj380 datafile 2023-12-01T06:35:18.741 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:18.741 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj381 datafile 2023-12-01T06:35:18.892 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:18.892 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj382 datafile 2023-12-01T06:35:19.042 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:19.043 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj383 datafile 2023-12-01T06:35:19.210 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:19.210 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj384 datafile 2023-12-01T06:35:19.353 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:19.353 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj385 datafile 2023-12-01T06:35:19.495 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:19.496 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj386 datafile 2023-12-01T06:35:19.638 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:19.638 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj387 datafile 2023-12-01T06:35:19.815 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:19.815 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj388 datafile 2023-12-01T06:35:19.974 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:19.974 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj389 datafile 2023-12-01T06:35:20.116 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:20.116 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj390 datafile 2023-12-01T06:35:20.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:20.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj391 datafile 2023-12-01T06:35:20.435 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:20.435 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj392 datafile 2023-12-01T06:35:20.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:20.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj393 datafile 2023-12-01T06:35:20.745 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:20.746 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj394 datafile 2023-12-01T06:35:20.891 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:20.892 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj395 datafile 2023-12-01T06:35:21.048 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:21.048 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj396 datafile 2023-12-01T06:35:21.215 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:21.215 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj397 datafile 2023-12-01T06:35:21.374 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:21.374 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj398 datafile 2023-12-01T06:35:21.541 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:21.542 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj399 datafile 2023-12-01T06:35:21.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:21.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj400 datafile 2023-12-01T06:35:21.853 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:21.853 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj401 datafile 2023-12-01T06:35:22.003 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:22.003 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj402 datafile 2023-12-01T06:35:22.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:22.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj403 datafile 2023-12-01T06:35:22.305 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:22.305 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj404 datafile 2023-12-01T06:35:22.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:22.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj405 datafile 2023-12-01T06:35:22.623 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:22.624 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj406 datafile 2023-12-01T06:35:22.784 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:22.784 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj407 datafile 2023-12-01T06:35:22.934 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:22.934 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj408 datafile 2023-12-01T06:35:23.076 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:23.077 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj409 datafile 2023-12-01T06:35:23.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:23.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj410 datafile 2023-12-01T06:35:23.386 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:23.387 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj411 datafile 2023-12-01T06:35:23.521 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:23.521 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj412 datafile 2023-12-01T06:35:23.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:23.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj413 datafile 2023-12-01T06:35:23.832 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:23.832 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj414 datafile 2023-12-01T06:35:24.016 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:24.016 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj415 datafile 2023-12-01T06:35:24.175 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:24.175 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj416 datafile 2023-12-01T06:35:24.326 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:24.326 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj417 datafile 2023-12-01T06:35:24.469 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:24.469 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj418 datafile 2023-12-01T06:35:24.628 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:24.628 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj419 datafile 2023-12-01T06:35:24.788 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:24.788 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj420 datafile 2023-12-01T06:35:24.947 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:24.947 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj421 datafile 2023-12-01T06:35:25.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:25.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj422 datafile 2023-12-01T06:35:25.265 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:25.265 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj423 datafile 2023-12-01T06:35:25.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:25.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj424 datafile 2023-12-01T06:35:25.558 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:25.558 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj425 datafile 2023-12-01T06:35:25.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:25.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj426 datafile 2023-12-01T06:35:25.928 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:25.929 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj427 datafile 2023-12-01T06:35:26.079 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:26.079 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj428 datafile 2023-12-01T06:35:26.246 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:26.246 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj429 datafile 2023-12-01T06:35:26.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:26.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj430 datafile 2023-12-01T06:35:26.565 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:26.565 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj431 datafile 2023-12-01T06:35:26.724 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:26.724 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj432 datafile 2023-12-01T06:35:26.875 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:26.875 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj433 datafile 2023-12-01T06:35:27.060 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:27.060 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj434 datafile 2023-12-01T06:35:27.238 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:27.238 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj435 datafile 2023-12-01T06:35:27.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:27.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj436 datafile 2023-12-01T06:35:27.571 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:27.572 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj437 datafile 2023-12-01T06:35:27.755 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:27.755 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj438 datafile 2023-12-01T06:35:27.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:27.924 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj439 datafile 2023-12-01T06:35:28.082 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:28.083 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj440 datafile 2023-12-01T06:35:28.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:28.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj441 datafile 2023-12-01T06:35:28.368 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:28.368 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj442 datafile 2023-12-01T06:35:28.511 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:28.511 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj443 datafile 2023-12-01T06:35:28.661 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:28.661 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj444 datafile 2023-12-01T06:35:28.812 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:28.812 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj445 datafile 2023-12-01T06:35:28.996 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:28.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj446 datafile 2023-12-01T06:35:29.256 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:29.257 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj447 datafile 2023-12-01T06:35:29.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:29.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj448 datafile 2023-12-01T06:35:29.566 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:29.566 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj449 datafile 2023-12-01T06:35:29.733 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:29.734 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj450 datafile 2023-12-01T06:35:29.885 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:29.885 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj451 datafile 2023-12-01T06:35:30.044 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:30.044 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj452 datafile 2023-12-01T06:35:30.195 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:30.195 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj453 datafile 2023-12-01T06:35:30.355 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:30.355 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj454 datafile 2023-12-01T06:35:30.497 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:30.497 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj455 datafile 2023-12-01T06:35:30.699 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:30.699 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj456 datafile 2023-12-01T06:35:30.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:30.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj457 datafile 2023-12-01T06:35:31.008 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:31.008 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj458 datafile 2023-12-01T06:35:31.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:31.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj459 datafile 2023-12-01T06:35:31.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:31.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj460 datafile 2023-12-01T06:35:31.494 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:31.494 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj461 datafile 2023-12-01T06:35:31.645 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:31.645 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj462 datafile 2023-12-01T06:35:31.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:31.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj463 datafile 2023-12-01T06:35:31.989 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:31.990 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj464 datafile 2023-12-01T06:35:32.140 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:32.140 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj465 datafile 2023-12-01T06:35:32.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:32.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj466 datafile 2023-12-01T06:35:32.434 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:32.434 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj467 datafile 2023-12-01T06:35:32.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:32.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj468 datafile 2023-12-01T06:35:32.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:32.761 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj469 datafile 2023-12-01T06:35:32.912 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:32.912 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj470 datafile 2023-12-01T06:35:33.088 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:33.088 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj471 datafile 2023-12-01T06:35:33.264 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:33.265 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj472 datafile 2023-12-01T06:35:33.423 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:33.423 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj473 datafile 2023-12-01T06:35:33.566 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:33.566 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj474 datafile 2023-12-01T06:35:33.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:33.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj475 datafile 2023-12-01T06:35:33.918 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:33.919 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj476 datafile 2023-12-01T06:35:34.060 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:34.060 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj477 datafile 2023-12-01T06:35:34.211 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:34.211 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj478 datafile 2023-12-01T06:35:34.630 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:34.630 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj479 datafile 2023-12-01T06:35:34.780 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:34.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj480 datafile 2023-12-01T06:35:34.931 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:34.931 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj481 datafile 2023-12-01T06:35:35.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:35.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj482 datafile 2023-12-01T06:35:35.261 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:35.261 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj483 datafile 2023-12-01T06:35:35.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:35.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj484 datafile 2023-12-01T06:35:35.568 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:35.569 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj485 datafile 2023-12-01T06:35:35.728 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:35.728 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj486 datafile 2023-12-01T06:35:35.904 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:35.904 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj487 datafile 2023-12-01T06:35:36.072 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:36.072 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj488 datafile 2023-12-01T06:35:36.231 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:36.232 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj489 datafile 2023-12-01T06:35:36.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:36.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj490 datafile 2023-12-01T06:35:36.543 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:36.543 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj491 datafile 2023-12-01T06:35:36.700 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:36.700 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj492 datafile 2023-12-01T06:35:36.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:36.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj493 datafile 2023-12-01T06:35:37.002 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:37.003 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj494 datafile 2023-12-01T06:35:37.145 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:37.145 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj495 datafile 2023-12-01T06:35:37.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:37.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj496 datafile 2023-12-01T06:35:37.448 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:37.448 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj497 datafile 2023-12-01T06:35:37.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:37.607 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj498 datafile 2023-12-01T06:35:37.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:37.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj499 datafile 2023-12-01T06:35:37.934 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:37.934 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj500 datafile 2023-12-01T06:35:38.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:38.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj501 datafile 2023-12-01T06:35:38.220 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:38.220 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj502 datafile 2023-12-01T06:35:38.370 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:38.370 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj503 datafile 2023-12-01T06:35:38.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:38.513 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj504 datafile 2023-12-01T06:35:38.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:38.664 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj505 datafile 2023-12-01T06:35:38.806 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:38.806 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj506 datafile 2023-12-01T06:35:38.957 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:38.957 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj507 datafile 2023-12-01T06:35:39.142 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:39.142 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj508 datafile 2023-12-01T06:35:39.292 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:39.293 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj509 datafile 2023-12-01T06:35:39.468 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:39.469 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj510 datafile 2023-12-01T06:35:39.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:39.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj511 datafile 2023-12-01T06:35:39.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:39.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj512 datafile 2023-12-01T06:35:40.031 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:40.031 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj513 datafile 2023-12-01T06:35:40.224 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:40.224 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj514 datafile 2023-12-01T06:35:40.374 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:40.374 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj515 datafile 2023-12-01T06:35:40.534 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:40.534 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj516 datafile 2023-12-01T06:35:40.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:40.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj517 datafile 2023-12-01T06:35:40.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:40.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj518 datafile 2023-12-01T06:35:40.977 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:40.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj519 datafile 2023-12-01T06:35:41.121 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:41.121 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj520 datafile 2023-12-01T06:35:41.297 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:41.297 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj521 datafile 2023-12-01T06:35:41.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:41.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj522 datafile 2023-12-01T06:35:41.607 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:41.607 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj523 datafile 2023-12-01T06:35:41.774 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:41.774 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj524 datafile 2023-12-01T06:35:41.951 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:41.951 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj525 datafile 2023-12-01T06:35:42.127 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:42.127 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj526 datafile 2023-12-01T06:35:42.269 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:42.269 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj527 datafile 2023-12-01T06:35:42.420 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:42.420 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj528 datafile 2023-12-01T06:35:42.580 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:42.580 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj529 datafile 2023-12-01T06:35:42.748 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:42.748 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj530 datafile 2023-12-01T06:35:42.915 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:42.915 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj531 datafile 2023-12-01T06:35:43.082 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:43.083 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj532 datafile 2023-12-01T06:35:43.258 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:43.258 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj533 datafile 2023-12-01T06:35:43.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:43.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj534 datafile 2023-12-01T06:35:43.560 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:43.561 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj535 datafile 2023-12-01T06:35:43.728 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:43.728 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj536 datafile 2023-12-01T06:35:43.914 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:43.914 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj537 datafile 2023-12-01T06:35:44.063 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:44.064 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj538 datafile 2023-12-01T06:35:44.223 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:44.223 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj539 datafile 2023-12-01T06:35:44.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:44.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj540 datafile 2023-12-01T06:35:44.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:44.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj541 datafile 2023-12-01T06:35:44.667 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:44.667 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj542 datafile 2023-12-01T06:35:44.859 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:44.860 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj543 datafile 2023-12-01T06:35:45.028 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:45.028 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj544 datafile 2023-12-01T06:35:45.195 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:45.195 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj545 datafile 2023-12-01T06:35:45.347 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:45.347 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj546 datafile 2023-12-01T06:35:45.506 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:45.506 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj547 datafile 2023-12-01T06:35:45.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:45.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj548 datafile 2023-12-01T06:35:45.875 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:45.875 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj549 datafile 2023-12-01T06:35:46.034 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:46.034 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj550 datafile 2023-12-01T06:35:46.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:46.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj551 datafile 2023-12-01T06:35:46.335 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:46.335 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj552 datafile 2023-12-01T06:35:46.478 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:46.478 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj553 datafile 2023-12-01T06:35:46.630 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:46.630 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj554 datafile 2023-12-01T06:35:46.780 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:46.780 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj555 datafile 2023-12-01T06:35:46.940 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:46.940 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj556 datafile 2023-12-01T06:35:47.107 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:47.107 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj557 datafile 2023-12-01T06:35:47.275 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:47.275 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj558 datafile 2023-12-01T06:35:47.426 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:47.426 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj559 datafile 2023-12-01T06:35:47.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:47.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj560 datafile 2023-12-01T06:35:47.770 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:47.770 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj561 datafile 2023-12-01T06:35:47.938 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:47.938 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj562 datafile 2023-12-01T06:35:48.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:48.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj563 datafile 2023-12-01T06:35:48.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:48.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj564 datafile 2023-12-01T06:35:48.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:48.383 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj565 datafile 2023-12-01T06:35:48.542 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:48.543 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj566 datafile 2023-12-01T06:35:48.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:48.718 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj567 datafile 2023-12-01T06:35:48.868 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:48.869 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj568 datafile 2023-12-01T06:35:49.011 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:49.011 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj569 datafile 2023-12-01T06:35:49.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:49.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj570 datafile 2023-12-01T06:35:49.330 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:49.330 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj571 datafile 2023-12-01T06:35:49.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:49.465 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj572 datafile 2023-12-01T06:35:49.614 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:49.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj573 datafile 2023-12-01T06:35:49.774 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:49.774 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj574 datafile 2023-12-01T06:35:49.958 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:49.959 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj575 datafile 2023-12-01T06:35:50.109 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:50.109 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj576 datafile 2023-12-01T06:35:50.261 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:50.261 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj577 datafile 2023-12-01T06:35:50.428 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:50.428 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj578 datafile 2023-12-01T06:35:50.570 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:50.571 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj579 datafile 2023-12-01T06:35:50.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:50.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj580 datafile 2023-12-01T06:35:50.872 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:50.872 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj581 datafile 2023-12-01T06:35:51.057 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:51.057 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj582 datafile 2023-12-01T06:35:51.243 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:51.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj583 datafile 2023-12-01T06:35:51.392 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:51.393 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj584 datafile 2023-12-01T06:35:51.560 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:51.561 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj585 datafile 2023-12-01T06:35:51.745 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:51.745 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj586 datafile 2023-12-01T06:35:51.913 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:51.913 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj587 datafile 2023-12-01T06:35:52.073 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:52.073 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj588 datafile 2023-12-01T06:35:52.216 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:52.216 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj589 datafile 2023-12-01T06:35:52.376 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:52.376 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj590 datafile 2023-12-01T06:35:52.526 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:52.526 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj591 datafile 2023-12-01T06:35:52.677 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:52.677 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj592 datafile 2023-12-01T06:35:52.829 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:52.829 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj593 datafile 2023-12-01T06:35:52.971 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:52.971 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj594 datafile 2023-12-01T06:35:53.115 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:53.115 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj595 datafile 2023-12-01T06:35:53.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:53.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj596 datafile 2023-12-01T06:35:53.433 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:53.433 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj597 datafile 2023-12-01T06:35:53.574 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:53.575 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj598 datafile 2023-12-01T06:35:53.734 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:53.734 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj599 datafile 2023-12-01T06:35:53.911 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:53.911 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj600 datafile 2023-12-01T06:35:54.078 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:54.078 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj601 datafile 2023-12-01T06:35:54.238 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:54.238 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj602 datafile 2023-12-01T06:35:54.388 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:54.388 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj603 datafile 2023-12-01T06:35:54.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:54.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj604 datafile 2023-12-01T06:35:54.692 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:54.692 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj605 datafile 2023-12-01T06:35:54.859 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:54.859 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj606 datafile 2023-12-01T06:35:55.001 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:55.001 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj607 datafile 2023-12-01T06:35:55.151 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:55.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj608 datafile 2023-12-01T06:35:55.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:55.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj609 datafile 2023-12-01T06:35:55.446 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:55.447 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj610 datafile 2023-12-01T06:35:55.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:55.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj611 datafile 2023-12-01T06:35:55.764 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:55.764 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj612 datafile 2023-12-01T06:35:55.940 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:55.940 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj613 datafile 2023-12-01T06:35:56.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:56.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj614 datafile 2023-12-01T06:35:56.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:56.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj615 datafile 2023-12-01T06:35:56.392 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:56.393 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj616 datafile 2023-12-01T06:35:56.544 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:56.544 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj617 datafile 2023-12-01T06:35:56.703 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:56.703 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj618 datafile 2023-12-01T06:35:56.863 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:56.863 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj619 datafile 2023-12-01T06:35:57.032 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:57.032 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj620 datafile 2023-12-01T06:35:57.198 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:57.199 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj621 datafile 2023-12-01T06:35:57.349 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:57.349 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj622 datafile 2023-12-01T06:35:57.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:57.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj623 datafile 2023-12-01T06:35:57.693 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:57.693 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj624 datafile 2023-12-01T06:35:57.861 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:57.861 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj625 datafile 2023-12-01T06:35:58.004 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:58.004 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj626 datafile 2023-12-01T06:35:58.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:58.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj627 datafile 2023-12-01T06:35:58.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:58.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj628 datafile 2023-12-01T06:35:58.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:58.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj629 datafile 2023-12-01T06:35:58.607 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:58.607 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj630 datafile 2023-12-01T06:35:58.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:58.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj631 datafile 2023-12-01T06:35:58.926 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:58.926 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj632 datafile 2023-12-01T06:35:59.084 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:59.085 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj633 datafile 2023-12-01T06:35:59.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:59.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj634 datafile 2023-12-01T06:35:59.412 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:59.412 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj635 datafile 2023-12-01T06:35:59.571 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:59.571 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj636 datafile 2023-12-01T06:35:59.730 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:59.730 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj637 datafile 2023-12-01T06:35:59.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:35:59.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj638 datafile 2023-12-01T06:36:00.116 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:00.116 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj639 datafile 2023-12-01T06:36:00.325 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:00.326 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj640 datafile 2023-12-01T06:36:00.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:00.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj641 datafile 2023-12-01T06:36:00.655 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:00.655 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj642 datafile 2023-12-01T06:36:00.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:00.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj643 datafile 2023-12-01T06:36:00.979 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:00.979 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj644 datafile 2023-12-01T06:36:01.138 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:01.138 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj645 datafile 2023-12-01T06:36:01.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:01.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj646 datafile 2023-12-01T06:36:01.465 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:01.465 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj647 datafile 2023-12-01T06:36:01.607 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:01.607 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj648 datafile 2023-12-01T06:36:01.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:01.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj649 datafile 2023-12-01T06:36:01.960 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:01.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj650 datafile 2023-12-01T06:36:02.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:02.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj651 datafile 2023-12-01T06:36:02.279 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:02.279 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj652 datafile 2023-12-01T06:36:02.446 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:02.446 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj653 datafile 2023-12-01T06:36:02.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:02.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj654 datafile 2023-12-01T06:36:02.790 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:02.790 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj655 datafile 2023-12-01T06:36:02.941 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:02.942 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj656 datafile 2023-12-01T06:36:03.118 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:03.119 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj657 datafile 2023-12-01T06:36:03.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:03.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj658 datafile 2023-12-01T06:36:03.435 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:03.435 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj659 datafile 2023-12-01T06:36:03.578 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:03.578 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj660 datafile 2023-12-01T06:36:03.738 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:03.738 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj661 datafile 2023-12-01T06:36:03.924 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:03.924 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj662 datafile 2023-12-01T06:36:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:04.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj663 datafile 2023-12-01T06:36:04.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:04.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj664 datafile 2023-12-01T06:36:04.426 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:04.426 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj665 datafile 2023-12-01T06:36:04.585 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:04.585 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj666 datafile 2023-12-01T06:36:04.736 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:04.736 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj667 datafile 2023-12-01T06:36:04.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:04.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj668 datafile 2023-12-01T06:36:05.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:05.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj669 datafile 2023-12-01T06:36:05.197 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:05.197 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj670 datafile 2023-12-01T06:36:05.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:05.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj671 datafile 2023-12-01T06:36:05.740 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:05.740 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj672 datafile 2023-12-01T06:36:05.926 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:05.926 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj673 datafile 2023-12-01T06:36:06.106 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:06.106 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj674 datafile 2023-12-01T06:36:06.260 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:06.261 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj675 datafile 2023-12-01T06:36:06.428 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:06.428 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj676 datafile 2023-12-01T06:36:06.579 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:06.579 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj677 datafile 2023-12-01T06:36:06.738 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:06.738 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj678 datafile 2023-12-01T06:36:06.898 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:06.898 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj679 datafile 2023-12-01T06:36:07.058 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:07.058 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj680 datafile 2023-12-01T06:36:07.216 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:07.216 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj681 datafile 2023-12-01T06:36:07.376 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:07.376 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj682 datafile 2023-12-01T06:36:07.543 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:07.544 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj683 datafile 2023-12-01T06:36:07.694 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:07.694 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj684 datafile 2023-12-01T06:36:07.855 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:07.855 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj685 datafile 2023-12-01T06:36:08.013 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:08.013 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj686 datafile 2023-12-01T06:36:08.155 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:08.156 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj687 datafile 2023-12-01T06:36:08.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:08.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj688 datafile 2023-12-01T06:36:08.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:08.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj689 datafile 2023-12-01T06:36:08.668 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:08.668 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj690 datafile 2023-12-01T06:36:08.817 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:08.818 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj691 datafile 2023-12-01T06:36:08.970 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:08.970 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj692 datafile 2023-12-01T06:36:09.145 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:09.145 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj693 datafile 2023-12-01T06:36:09.296 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:09.296 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj694 datafile 2023-12-01T06:36:09.455 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:09.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj695 datafile 2023-12-01T06:36:09.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:09.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj696 datafile 2023-12-01T06:36:09.774 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:09.775 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj697 datafile 2023-12-01T06:36:09.967 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:09.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj698 datafile 2023-12-01T06:36:10.134 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:10.134 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj699 datafile 2023-12-01T06:36:10.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:10.277 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj700 datafile 2023-12-01T06:36:10.436 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:10.436 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj701 datafile 2023-12-01T06:36:10.587 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:10.587 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj702 datafile 2023-12-01T06:36:10.830 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:10.830 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj703 datafile 2023-12-01T06:36:10.973 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:10.973 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj704 datafile 2023-12-01T06:36:11.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:11.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj705 datafile 2023-12-01T06:36:11.324 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:11.325 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj706 datafile 2023-12-01T06:36:11.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:11.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj707 datafile 2023-12-01T06:36:11.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:11.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj708 datafile 2023-12-01T06:36:11.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:11.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj709 datafile 2023-12-01T06:36:11.970 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:11.970 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj710 datafile 2023-12-01T06:36:12.154 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:12.154 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj711 datafile 2023-12-01T06:36:12.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:12.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj712 datafile 2023-12-01T06:36:12.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:12.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj713 datafile 2023-12-01T06:36:12.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:12.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj714 datafile 2023-12-01T06:36:12.766 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:12.766 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj715 datafile 2023-12-01T06:36:12.917 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:12.917 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj716 datafile 2023-12-01T06:36:13.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:13.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj717 datafile 2023-12-01T06:36:13.232 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:13.232 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj718 datafile 2023-12-01T06:36:13.379 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:13.379 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj719 datafile 2023-12-01T06:36:13.546 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:13.546 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj720 datafile 2023-12-01T06:36:13.697 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:13.697 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj721 datafile 2023-12-01T06:36:13.866 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:13.866 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj722 datafile 2023-12-01T06:36:14.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:14.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj723 datafile 2023-12-01T06:36:14.200 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:14.201 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj724 datafile 2023-12-01T06:36:14.359 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:14.360 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj725 datafile 2023-12-01T06:36:14.519 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:14.519 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj726 datafile 2023-12-01T06:36:14.670 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:14.670 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj727 datafile 2023-12-01T06:36:14.821 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:14.821 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj728 datafile 2023-12-01T06:36:14.971 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:14.972 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj729 datafile 2023-12-01T06:36:15.139 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:15.139 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj730 datafile 2023-12-01T06:36:15.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:15.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj731 datafile 2023-12-01T06:36:15.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:15.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj732 datafile 2023-12-01T06:36:15.625 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:15.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj733 datafile 2023-12-01T06:36:15.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:15.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj734 datafile 2023-12-01T06:36:15.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:15.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj735 datafile 2023-12-01T06:36:16.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:16.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj736 datafile 2023-12-01T06:36:16.296 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:16.297 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj737 datafile 2023-12-01T06:36:16.447 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:16.447 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj738 datafile 2023-12-01T06:36:16.599 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:16.599 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj739 datafile 2023-12-01T06:36:16.749 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:16.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj740 datafile 2023-12-01T06:36:16.909 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:16.909 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj741 datafile 2023-12-01T06:36:17.059 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:17.059 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj742 datafile 2023-12-01T06:36:17.202 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:17.202 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj743 datafile 2023-12-01T06:36:17.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:17.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj744 datafile 2023-12-01T06:36:17.505 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:17.505 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj745 datafile 2023-12-01T06:36:17.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:17.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj746 datafile 2023-12-01T06:36:17.840 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:17.840 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj747 datafile 2023-12-01T06:36:17.999 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:17.999 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj748 datafile 2023-12-01T06:36:18.159 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:18.159 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj749 datafile 2023-12-01T06:36:18.310 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:18.310 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj750 datafile 2023-12-01T06:36:18.461 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:18.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj751 datafile 2023-12-01T06:36:18.611 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:18.611 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj752 datafile 2023-12-01T06:36:18.779 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:18.779 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj753 datafile 2023-12-01T06:36:18.930 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:18.930 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj754 datafile 2023-12-01T06:36:19.098 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:19.098 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj755 datafile 2023-12-01T06:36:19.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:19.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj756 datafile 2023-12-01T06:36:19.449 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:19.449 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj757 datafile 2023-12-01T06:36:19.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:19.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj758 datafile 2023-12-01T06:36:19.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:19.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj759 datafile 2023-12-01T06:36:19.953 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:19.953 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj760 datafile 2023-12-01T06:36:20.104 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:20.104 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj761 datafile 2023-12-01T06:36:20.271 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:20.272 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj762 datafile 2023-12-01T06:36:20.430 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:20.431 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj763 datafile 2023-12-01T06:36:20.590 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:20.590 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj764 datafile 2023-12-01T06:36:20.740 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:20.740 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj765 datafile 2023-12-01T06:36:20.900 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:20.900 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj766 datafile 2023-12-01T06:36:21.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:21.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj767 datafile 2023-12-01T06:36:21.227 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:21.227 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj768 datafile 2023-12-01T06:36:21.387 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:21.387 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj769 datafile 2023-12-01T06:36:21.554 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:21.554 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj770 datafile 2023-12-01T06:36:21.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:21.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj771 datafile 2023-12-01T06:36:21.916 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:21.916 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj772 datafile 2023-12-01T06:36:22.074 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:22.074 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj773 datafile 2023-12-01T06:36:22.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:22.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj774 datafile 2023-12-01T06:36:22.393 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:22.393 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj775 datafile 2023-12-01T06:36:22.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:22.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj776 datafile 2023-12-01T06:36:22.712 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:22.712 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj777 datafile 2023-12-01T06:36:22.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:22.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj778 datafile 2023-12-01T06:36:23.047 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:23.047 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj779 datafile 2023-12-01T06:36:23.214 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:23.214 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj780 datafile 2023-12-01T06:36:23.365 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:23.365 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj781 datafile 2023-12-01T06:36:23.516 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:23.516 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj782 datafile 2023-12-01T06:36:23.658 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:23.658 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj783 datafile 2023-12-01T06:36:23.826 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:23.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj784 datafile 2023-12-01T06:36:23.977 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:23.977 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj785 datafile 2023-12-01T06:36:24.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:24.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj786 datafile 2023-12-01T06:36:24.304 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:24.305 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj787 datafile 2023-12-01T06:36:24.440 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:24.440 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj788 datafile 2023-12-01T06:36:24.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:24.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj789 datafile 2023-12-01T06:36:24.766 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:24.766 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj790 datafile 2023-12-01T06:36:24.917 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:24.917 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj791 datafile 2023-12-01T06:36:25.084 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:25.084 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj792 datafile 2023-12-01T06:36:25.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:25.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj793 datafile 2023-12-01T06:36:25.436 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:25.436 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj794 datafile 2023-12-01T06:36:25.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:25.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj795 datafile 2023-12-01T06:36:25.771 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:25.771 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj796 datafile 2023-12-01T06:36:25.939 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:25.939 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj797 datafile 2023-12-01T06:36:26.107 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:26.107 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj798 datafile 2023-12-01T06:36:26.258 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:26.258 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj799 datafile 2023-12-01T06:36:26.426 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:26.426 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj800 datafile 2023-12-01T06:36:26.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:26.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj801 datafile 2023-12-01T06:36:26.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:26.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj802 datafile 2023-12-01T06:36:26.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:26.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj803 datafile 2023-12-01T06:36:27.062 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:27.062 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj804 datafile 2023-12-01T06:36:27.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:27.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj805 datafile 2023-12-01T06:36:27.390 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:27.390 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj806 datafile 2023-12-01T06:36:27.558 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:27.558 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj807 datafile 2023-12-01T06:36:27.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:27.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj808 datafile 2023-12-01T06:36:27.928 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:27.928 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj809 datafile 2023-12-01T06:36:28.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:28.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj810 datafile 2023-12-01T06:36:28.270 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:28.271 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj811 datafile 2023-12-01T06:36:28.438 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:28.438 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj812 datafile 2023-12-01T06:36:28.597 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:28.597 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj813 datafile 2023-12-01T06:36:28.740 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:28.741 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj814 datafile 2023-12-01T06:36:28.916 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:28.916 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj815 datafile 2023-12-01T06:36:29.082 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:29.083 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj816 datafile 2023-12-01T06:36:29.279 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:29.280 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj817 datafile 2023-12-01T06:36:29.444 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:29.444 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj818 datafile 2023-12-01T06:36:29.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:29.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj819 datafile 2023-12-01T06:36:29.804 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:29.804 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj820 datafile 2023-12-01T06:36:29.963 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:29.963 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj821 datafile 2023-12-01T06:36:30.181 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:30.181 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj822 datafile 2023-12-01T06:36:30.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:30.332 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj823 datafile 2023-12-01T06:36:30.492 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:30.492 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj824 datafile 2023-12-01T06:36:30.642 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:30.642 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj825 datafile 2023-12-01T06:36:30.826 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:30.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj826 datafile 2023-12-01T06:36:30.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:30.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj827 datafile 2023-12-01T06:36:31.229 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:31.230 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj828 datafile 2023-12-01T06:36:31.388 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:31.388 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj829 datafile 2023-12-01T06:36:31.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:31.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj830 datafile 2023-12-01T06:36:31.681 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:31.682 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj831 datafile 2023-12-01T06:36:31.934 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:31.934 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj832 datafile 2023-12-01T06:36:32.084 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:32.084 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj833 datafile 2023-12-01T06:36:32.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:32.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj834 datafile 2023-12-01T06:36:32.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:32.395 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj835 datafile 2023-12-01T06:36:32.562 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:32.562 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj836 datafile 2023-12-01T06:36:32.713 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:32.713 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj837 datafile 2023-12-01T06:36:32.872 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:32.872 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj838 datafile 2023-12-01T06:36:33.006 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:33.007 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj839 datafile 2023-12-01T06:36:33.183 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:33.183 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj840 datafile 2023-12-01T06:36:33.352 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:33.352 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj841 datafile 2023-12-01T06:36:33.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:33.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj842 datafile 2023-12-01T06:36:33.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:33.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj843 datafile 2023-12-01T06:36:33.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:33.812 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj844 datafile 2023-12-01T06:36:33.972 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:33.972 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj845 datafile 2023-12-01T06:36:34.131 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:34.131 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj846 datafile 2023-12-01T06:36:34.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:34.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj847 datafile 2023-12-01T06:36:34.416 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:34.416 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj848 datafile 2023-12-01T06:36:34.574 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:34.575 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj849 datafile 2023-12-01T06:36:34.735 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:34.735 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj850 datafile 2023-12-01T06:36:34.876 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:34.876 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj851 datafile 2023-12-01T06:36:35.044 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:35.045 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj852 datafile 2023-12-01T06:36:35.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:35.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj853 datafile 2023-12-01T06:36:35.372 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:35.372 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj854 datafile 2023-12-01T06:36:35.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:35.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj855 datafile 2023-12-01T06:36:35.681 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:35.682 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj856 datafile 2023-12-01T06:36:35.826 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:35.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj857 datafile 2023-12-01T06:36:35.994 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:35.994 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj858 datafile 2023-12-01T06:36:36.145 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:36.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj859 datafile 2023-12-01T06:36:36.569 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:36.569 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj860 datafile 2023-12-01T06:36:36.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:36.725 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj861 datafile 2023-12-01T06:36:36.873 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:36.873 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj862 datafile 2023-12-01T06:36:37.032 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:37.032 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj863 datafile 2023-12-01T06:36:37.183 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:37.183 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj864 datafile 2023-12-01T06:36:37.334 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:37.334 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj865 datafile 2023-12-01T06:36:37.493 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:37.494 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj866 datafile 2023-12-01T06:36:37.645 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:37.645 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj867 datafile 2023-12-01T06:36:37.803 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:37.804 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj868 datafile 2023-12-01T06:36:37.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:37.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj869 datafile 2023-12-01T06:36:38.106 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:38.106 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj870 datafile 2023-12-01T06:36:38.265 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:38.265 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj871 datafile 2023-12-01T06:36:38.433 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:38.433 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj872 datafile 2023-12-01T06:36:38.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:38.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj873 datafile 2023-12-01T06:36:38.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:38.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj874 datafile 2023-12-01T06:36:38.902 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:38.902 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj875 datafile 2023-12-01T06:36:39.061 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:39.061 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj876 datafile 2023-12-01T06:36:39.229 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:39.229 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj877 datafile 2023-12-01T06:36:39.405 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:39.405 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj878 datafile 2023-12-01T06:36:39.565 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:39.565 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj879 datafile 2023-12-01T06:36:39.707 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:39.707 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj880 datafile 2023-12-01T06:36:39.873 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:39.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj881 datafile 2023-12-01T06:36:40.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:40.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj882 datafile 2023-12-01T06:36:40.168 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:40.168 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj883 datafile 2023-12-01T06:36:40.318 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:40.319 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj884 datafile 2023-12-01T06:36:40.469 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:40.469 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj885 datafile 2023-12-01T06:36:40.621 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:40.621 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj886 datafile 2023-12-01T06:36:40.789 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:40.789 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj887 datafile 2023-12-01T06:36:40.938 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:40.939 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj888 datafile 2023-12-01T06:36:41.081 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:41.081 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj889 datafile 2023-12-01T06:36:41.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:41.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj890 datafile 2023-12-01T06:36:41.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:41.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj891 datafile 2023-12-01T06:36:41.559 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:41.559 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj892 datafile 2023-12-01T06:36:41.835 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:41.835 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj893 datafile 2023-12-01T06:36:41.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:41.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj894 datafile 2023-12-01T06:36:42.187 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:42.187 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj895 datafile 2023-12-01T06:36:42.347 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:42.347 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj896 datafile 2023-12-01T06:36:42.489 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:42.489 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj897 datafile 2023-12-01T06:36:42.641 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:42.641 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj898 datafile 2023-12-01T06:36:42.791 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:42.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj899 datafile 2023-12-01T06:36:42.951 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:42.951 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj900 datafile 2023-12-01T06:36:43.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:43.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj901 datafile 2023-12-01T06:36:43.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:43.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj902 datafile 2023-12-01T06:36:43.438 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:43.438 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj903 datafile 2023-12-01T06:36:43.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:43.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj904 datafile 2023-12-01T06:36:43.747 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:43.747 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj905 datafile 2023-12-01T06:36:43.915 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:43.915 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj906 datafile 2023-12-01T06:36:44.083 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:44.083 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj907 datafile 2023-12-01T06:36:44.241 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:44.241 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj908 datafile 2023-12-01T06:36:44.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:44.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj909 datafile 2023-12-01T06:36:44.535 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:44.535 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj910 datafile 2023-12-01T06:36:44.686 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:44.686 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj911 datafile 2023-12-01T06:36:44.836 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:44.837 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj912 datafile 2023-12-01T06:36:45.004 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:45.004 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj913 datafile 2023-12-01T06:36:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:45.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj914 datafile 2023-12-01T06:36:45.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:45.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj915 datafile 2023-12-01T06:36:45.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:45.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj916 datafile 2023-12-01T06:36:45.633 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:45.633 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj917 datafile 2023-12-01T06:36:45.801 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:45.801 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj918 datafile 2023-12-01T06:36:45.960 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:45.960 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj919 datafile 2023-12-01T06:36:46.111 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:46.111 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj920 datafile 2023-12-01T06:36:46.254 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:46.254 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj921 datafile 2023-12-01T06:36:46.413 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:46.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj922 datafile 2023-12-01T06:36:46.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:46.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj923 datafile 2023-12-01T06:36:46.707 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:46.707 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj924 datafile 2023-12-01T06:36:46.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:46.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj925 datafile 2023-12-01T06:36:47.009 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:47.009 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj926 datafile 2023-12-01T06:36:47.167 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:47.167 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj927 datafile 2023-12-01T06:36:47.310 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:47.310 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj928 datafile 2023-12-01T06:36:47.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:47.461 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj929 datafile 2023-12-01T06:36:47.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:47.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj930 datafile 2023-12-01T06:36:47.819 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:47.819 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj931 datafile 2023-12-01T06:36:47.981 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:47.981 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj932 datafile 2023-12-01T06:36:48.157 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:48.157 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj933 datafile 2023-12-01T06:36:48.341 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:48.341 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj934 datafile 2023-12-01T06:36:48.500 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:48.500 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj935 datafile 2023-12-01T06:36:48.669 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:48.669 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj936 datafile 2023-12-01T06:36:48.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:48.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj937 datafile 2023-12-01T06:36:48.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:48.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj938 datafile 2023-12-01T06:36:49.154 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:49.154 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj939 datafile 2023-12-01T06:36:49.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:49.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj940 datafile 2023-12-01T06:36:49.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:49.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj941 datafile 2023-12-01T06:36:49.614 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:49.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj942 datafile 2023-12-01T06:36:49.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:49.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj943 datafile 2023-12-01T06:36:49.967 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:49.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj944 datafile 2023-12-01T06:36:50.135 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:50.135 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj945 datafile 2023-12-01T06:36:50.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:50.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj946 datafile 2023-12-01T06:36:50.428 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:50.428 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj947 datafile 2023-12-01T06:36:50.613 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:50.613 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj948 datafile 2023-12-01T06:36:50.772 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:50.772 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj949 datafile 2023-12-01T06:36:50.931 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:50.931 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj950 datafile 2023-12-01T06:36:51.082 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:51.082 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj951 datafile 2023-12-01T06:36:51.260 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:51.260 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj952 datafile 2023-12-01T06:36:51.410 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:51.410 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj953 datafile 2023-12-01T06:36:51.561 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:51.561 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj954 datafile 2023-12-01T06:36:51.711 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:51.711 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj955 datafile 2023-12-01T06:36:51.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:51.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj956 datafile 2023-12-01T06:36:52.022 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:52.022 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj957 datafile 2023-12-01T06:36:52.164 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:52.164 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj958 datafile 2023-12-01T06:36:52.306 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:52.306 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj959 datafile 2023-12-01T06:36:52.465 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:52.465 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj960 datafile 2023-12-01T06:36:52.633 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:52.634 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj961 datafile 2023-12-01T06:36:52.784 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:52.784 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj962 datafile 2023-12-01T06:36:52.944 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:52.944 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj963 datafile 2023-12-01T06:36:53.094 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:53.095 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj964 datafile 2023-12-01T06:36:53.262 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:53.262 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj965 datafile 2023-12-01T06:36:53.422 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:53.422 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj966 datafile 2023-12-01T06:36:53.572 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:53.572 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj967 datafile 2023-12-01T06:36:53.726 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:53.727 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj968 datafile 2023-12-01T06:36:53.934 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:53.934 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj969 datafile 2023-12-01T06:36:54.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:54.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj970 datafile 2023-12-01T06:36:54.252 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:54.252 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj971 datafile 2023-12-01T06:36:54.411 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:54.412 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj972 datafile 2023-12-01T06:36:54.561 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:54.562 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj973 datafile 2023-12-01T06:36:54.704 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:54.704 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj974 datafile 2023-12-01T06:36:54.855 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:54.855 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj975 datafile 2023-12-01T06:36:54.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:54.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj976 datafile 2023-12-01T06:36:55.140 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:55.141 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj977 datafile 2023-12-01T06:36:55.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:55.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj978 datafile 2023-12-01T06:36:55.467 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:55.468 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj979 datafile 2023-12-01T06:36:55.609 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:55.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj980 datafile 2023-12-01T06:36:55.776 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:55.776 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj981 datafile 2023-12-01T06:36:55.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:55.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj982 datafile 2023-12-01T06:36:56.096 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:56.096 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj983 datafile 2023-12-01T06:36:56.272 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:56.272 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj984 datafile 2023-12-01T06:36:56.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:56.415 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj985 datafile 2023-12-01T06:36:56.574 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:56.574 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj986 datafile 2023-12-01T06:36:56.733 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:56.734 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj987 datafile 2023-12-01T06:36:56.892 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:56.892 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj988 datafile 2023-12-01T06:36:57.060 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:57.061 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj989 datafile 2023-12-01T06:36:57.245 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:57.245 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj990 datafile 2023-12-01T06:36:57.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:57.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj991 datafile 2023-12-01T06:36:57.563 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:57.563 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj992 datafile 2023-12-01T06:36:57.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:57.718 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj993 datafile 2023-12-01T06:36:57.883 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:57.883 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj994 datafile 2023-12-01T06:36:58.033 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:58.033 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj995 datafile 2023-12-01T06:36:58.191 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:58.192 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj996 datafile 2023-12-01T06:36:58.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:58.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj997 datafile 2023-12-01T06:36:58.519 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:58.519 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj998 datafile 2023-12-01T06:36:58.670 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:58.670 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj999 datafile 2023-12-01T06:36:58.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:58.821 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test1 put obj1000 datafile 2023-12-01T06:36:58.980 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:74: TEST_recover_unexpected: for i in $(seq 1 $POOLS) 2023-12-01T06:36:58.981 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: seq 1 1000 2023-12-01T06:36:58.984 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:58.984 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj1 datafile 2023-12-01T06:36:59.149 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:59.149 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj2 datafile 2023-12-01T06:36:59.323 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:59.324 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj3 datafile 2023-12-01T06:36:59.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:59.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj4 datafile 2023-12-01T06:36:59.642 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:59.642 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj5 datafile 2023-12-01T06:36:59.817 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:59.817 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj6 datafile 2023-12-01T06:36:59.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:36:59.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj7 datafile 2023-12-01T06:37:00.137 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:00.137 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj8 datafile 2023-12-01T06:37:00.308 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:00.308 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj9 datafile 2023-12-01T06:37:00.455 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:00.455 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj10 datafile 2023-12-01T06:37:00.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:00.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj11 datafile 2023-12-01T06:37:00.757 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:00.757 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj12 datafile 2023-12-01T06:37:00.908 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:00.908 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj13 datafile 2023-12-01T06:37:01.051 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:01.051 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj14 datafile 2023-12-01T06:37:01.210 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:01.210 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj15 datafile 2023-12-01T06:37:01.369 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:01.370 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj16 datafile 2023-12-01T06:37:01.529 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:01.529 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj17 datafile 2023-12-01T06:37:01.671 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:01.671 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj18 datafile 2023-12-01T06:37:01.873 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:01.873 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj19 datafile 2023-12-01T06:37:02.031 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:02.032 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj20 datafile 2023-12-01T06:37:02.290 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:02.290 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj21 datafile 2023-12-01T06:37:02.450 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:02.450 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj22 datafile 2023-12-01T06:37:02.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:02.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj23 datafile 2023-12-01T06:37:02.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:02.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj24 datafile 2023-12-01T06:37:02.944 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:02.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj25 datafile 2023-12-01T06:37:03.095 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:03.096 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj26 datafile 2023-12-01T06:37:03.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:03.289 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj27 datafile 2023-12-01T06:37:03.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:03.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj28 datafile 2023-12-01T06:37:03.625 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:03.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj29 datafile 2023-12-01T06:37:03.799 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:03.800 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj30 datafile 2023-12-01T06:37:03.975 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:03.975 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj31 datafile 2023-12-01T06:37:04.135 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:04.135 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj32 datafile 2023-12-01T06:37:04.303 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:04.303 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj33 datafile 2023-12-01T06:37:04.470 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:04.470 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj34 datafile 2023-12-01T06:37:04.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:04.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj35 datafile 2023-12-01T06:37:04.771 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:04.771 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj36 datafile 2023-12-01T06:37:04.930 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:04.931 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj37 datafile 2023-12-01T06:37:05.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:05.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj38 datafile 2023-12-01T06:37:05.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:05.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj39 datafile 2023-12-01T06:37:05.400 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:05.400 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj40 datafile 2023-12-01T06:37:05.560 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:05.560 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj41 datafile 2023-12-01T06:37:05.711 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:05.711 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj42 datafile 2023-12-01T06:37:05.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:05.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj43 datafile 2023-12-01T06:37:06.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:06.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj44 datafile 2023-12-01T06:37:06.214 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:06.214 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj45 datafile 2023-12-01T06:37:06.365 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:06.365 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj46 datafile 2023-12-01T06:37:06.524 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:06.524 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj47 datafile 2023-12-01T06:37:06.686 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:06.687 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj48 datafile 2023-12-01T06:37:06.834 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:06.834 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj49 datafile 2023-12-01T06:37:06.985 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:06.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj50 datafile 2023-12-01T06:37:07.136 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:07.136 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj51 datafile 2023-12-01T06:37:07.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:07.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj52 datafile 2023-12-01T06:37:07.772 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:07.772 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj53 datafile 2023-12-01T06:37:07.932 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:07.932 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj54 datafile 2023-12-01T06:37:08.082 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:08.082 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj55 datafile 2023-12-01T06:37:08.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:08.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj56 datafile 2023-12-01T06:37:08.392 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:08.392 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj57 datafile 2023-12-01T06:37:08.552 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:08.552 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj58 datafile 2023-12-01T06:37:08.694 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:08.694 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj59 datafile 2023-12-01T06:37:08.854 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:08.854 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj60 datafile 2023-12-01T06:37:09.004 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:09.005 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj61 datafile 2023-12-01T06:37:09.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:09.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj62 datafile 2023-12-01T06:37:09.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:09.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj63 datafile 2023-12-01T06:37:09.507 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:09.507 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj64 datafile 2023-12-01T06:37:09.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:09.667 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj65 datafile 2023-12-01T06:37:09.869 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:09.869 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj66 datafile 2023-12-01T06:37:10.044 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:10.044 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj67 datafile 2023-12-01T06:37:10.203 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:10.203 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj68 datafile 2023-12-01T06:37:10.378 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:10.378 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj69 datafile 2023-12-01T06:37:10.529 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:10.529 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj70 datafile 2023-12-01T06:37:10.681 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:10.681 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj71 datafile 2023-12-01T06:37:10.823 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:10.823 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj72 datafile 2023-12-01T06:37:10.983 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:10.983 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj73 datafile 2023-12-01T06:37:11.141 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:11.141 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj74 datafile 2023-12-01T06:37:11.309 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:11.309 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj75 datafile 2023-12-01T06:37:11.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:11.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj76 datafile 2023-12-01T06:37:11.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:11.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj77 datafile 2023-12-01T06:37:11.786 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:11.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj78 datafile 2023-12-01T06:37:11.964 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:11.964 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj79 datafile 2023-12-01T06:37:12.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:12.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj80 datafile 2023-12-01T06:37:12.300 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:12.300 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj81 datafile 2023-12-01T06:37:12.441 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:12.441 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj82 datafile 2023-12-01T06:37:12.711 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:12.712 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj83 datafile 2023-12-01T06:37:12.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:12.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj84 datafile 2023-12-01T06:37:13.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:13.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj85 datafile 2023-12-01T06:37:13.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:13.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj86 datafile 2023-12-01T06:37:13.371 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:13.372 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj87 datafile 2023-12-01T06:37:13.538 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:13.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj88 datafile 2023-12-01T06:37:13.689 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:13.689 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj89 datafile 2023-12-01T06:37:13.841 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:13.841 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj90 datafile 2023-12-01T06:37:14.000 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:14.000 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj91 datafile 2023-12-01T06:37:14.159 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:14.159 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj92 datafile 2023-12-01T06:37:14.326 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:14.327 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj93 datafile 2023-12-01T06:37:14.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:14.478 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj94 datafile 2023-12-01T06:37:14.629 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:14.629 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj95 datafile 2023-12-01T06:37:14.788 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:14.788 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj96 datafile 2023-12-01T06:37:14.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:14.947 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj97 datafile 2023-12-01T06:37:15.106 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:15.107 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj98 datafile 2023-12-01T06:37:15.299 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:15.299 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj99 datafile 2023-12-01T06:37:15.450 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:15.450 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj100 datafile 2023-12-01T06:37:15.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:15.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj101 datafile 2023-12-01T06:37:15.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:15.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj102 datafile 2023-12-01T06:37:15.988 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:15.988 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj103 datafile 2023-12-01T06:37:16.137 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:16.137 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj104 datafile 2023-12-01T06:37:16.280 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:16.280 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj105 datafile 2023-12-01T06:37:16.447 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:16.447 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj106 datafile 2023-12-01T06:37:16.600 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:16.600 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj107 datafile 2023-12-01T06:37:16.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:16.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj108 datafile 2023-12-01T06:37:16.917 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:16.918 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj109 datafile 2023-12-01T06:37:17.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:17.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj110 datafile 2023-12-01T06:37:17.265 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:17.265 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj111 datafile 2023-12-01T06:37:17.419 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:17.419 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj112 datafile 2023-12-01T06:37:17.570 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:17.570 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj113 datafile 2023-12-01T06:37:17.763 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:17.763 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj114 datafile 2023-12-01T06:37:17.940 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:17.940 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj115 datafile 2023-12-01T06:37:18.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:18.090 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj116 datafile 2023-12-01T06:37:18.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:18.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj117 datafile 2023-12-01T06:37:18.434 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:18.434 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj118 datafile 2023-12-01T06:37:18.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:18.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj119 datafile 2023-12-01T06:37:18.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:18.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj120 datafile 2023-12-01T06:37:18.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:18.979 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj121 datafile 2023-12-01T06:37:19.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:19.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj122 datafile 2023-12-01T06:37:19.297 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:19.297 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj123 datafile 2023-12-01T06:37:19.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:19.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj124 datafile 2023-12-01T06:37:19.623 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:19.624 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj125 datafile 2023-12-01T06:37:19.799 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:19.799 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj126 datafile 2023-12-01T06:37:19.975 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:19.976 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj127 datafile 2023-12-01T06:37:20.126 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:20.126 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj128 datafile 2023-12-01T06:37:20.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:20.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj129 datafile 2023-12-01T06:37:20.447 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:20.447 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj130 datafile 2023-12-01T06:37:20.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:20.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj131 datafile 2023-12-01T06:37:20.732 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:20.732 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj132 datafile 2023-12-01T06:37:20.883 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:20.883 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj133 datafile 2023-12-01T06:37:21.050 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:21.051 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj134 datafile 2023-12-01T06:37:21.235 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:21.235 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj135 datafile 2023-12-01T06:37:21.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:21.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj136 datafile 2023-12-01T06:37:21.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:21.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj137 datafile 2023-12-01T06:37:21.772 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:21.773 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj138 datafile 2023-12-01T06:37:21.949 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:21.949 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj139 datafile 2023-12-01T06:37:22.099 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:22.099 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj140 datafile 2023-12-01T06:37:22.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:22.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj141 datafile 2023-12-01T06:37:22.417 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:22.417 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj142 datafile 2023-12-01T06:37:22.568 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:22.568 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj143 datafile 2023-12-01T06:37:22.736 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:22.736 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj144 datafile 2023-12-01T06:37:22.897 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:22.898 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj145 datafile 2023-12-01T06:37:23.063 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:23.063 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj146 datafile 2023-12-01T06:37:23.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:23.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj147 datafile 2023-12-01T06:37:23.398 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:23.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj148 datafile 2023-12-01T06:37:23.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:23.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj149 datafile 2023-12-01T06:37:23.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:23.718 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj150 datafile 2023-12-01T06:37:23.876 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:23.876 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj151 datafile 2023-12-01T06:37:24.035 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:24.035 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj152 datafile 2023-12-01T06:37:24.233 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:24.233 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj153 datafile 2023-12-01T06:37:24.412 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:24.413 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj154 datafile 2023-12-01T06:37:24.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:24.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj155 datafile 2023-12-01T06:37:24.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:24.723 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj156 datafile 2023-12-01T06:37:24.873 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:24.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj157 datafile 2023-12-01T06:37:25.024 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:25.024 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj158 datafile 2023-12-01T06:37:25.176 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:25.176 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj159 datafile 2023-12-01T06:37:25.318 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:25.318 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj160 datafile 2023-12-01T06:37:25.485 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:25.485 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj161 datafile 2023-12-01T06:37:25.636 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:25.636 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj162 datafile 2023-12-01T06:37:25.804 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:25.804 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj163 datafile 2023-12-01T06:37:25.972 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:25.973 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj164 datafile 2023-12-01T06:37:26.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:26.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj165 datafile 2023-12-01T06:37:26.273 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:26.273 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj166 datafile 2023-12-01T06:37:26.441 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:26.441 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj167 datafile 2023-12-01T06:37:26.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:26.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj168 datafile 2023-12-01T06:37:26.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:26.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj169 datafile 2023-12-01T06:37:26.911 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:26.911 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj170 datafile 2023-12-01T06:37:27.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:27.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj171 datafile 2023-12-01T06:37:27.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:27.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj172 datafile 2023-12-01T06:37:27.455 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:27.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj173 datafile 2023-12-01T06:37:27.625 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:27.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj174 datafile 2023-12-01T06:37:27.808 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:27.808 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj175 datafile 2023-12-01T06:37:28.037 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:28.037 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj176 datafile 2023-12-01T06:37:28.194 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:28.195 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj177 datafile 2023-12-01T06:37:28.353 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:28.353 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj178 datafile 2023-12-01T06:37:28.496 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:28.496 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj179 datafile 2023-12-01T06:37:28.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:28.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj180 datafile 2023-12-01T06:37:28.815 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:28.815 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj181 datafile 2023-12-01T06:37:28.982 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:28.982 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj182 datafile 2023-12-01T06:37:29.140 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:29.141 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj183 datafile 2023-12-01T06:37:29.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:29.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj184 datafile 2023-12-01T06:37:29.467 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:29.467 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj185 datafile 2023-12-01T06:37:29.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:29.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj186 datafile 2023-12-01T06:37:29.777 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:29.777 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj187 datafile 2023-12-01T06:37:29.963 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:29.963 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj188 datafile 2023-12-01T06:37:30.113 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:30.113 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj189 datafile 2023-12-01T06:37:30.273 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:30.273 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj190 datafile 2023-12-01T06:37:30.440 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:30.441 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj191 datafile 2023-12-01T06:37:30.599 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:30.600 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj192 datafile 2023-12-01T06:37:30.751 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:30.751 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj193 datafile 2023-12-01T06:37:30.902 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:30.902 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj194 datafile 2023-12-01T06:37:31.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:31.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj195 datafile 2023-12-01T06:37:31.246 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:31.246 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj196 datafile 2023-12-01T06:37:31.422 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:31.422 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj197 datafile 2023-12-01T06:37:31.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:31.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj198 datafile 2023-12-01T06:37:31.715 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:31.715 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj199 datafile 2023-12-01T06:37:31.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:31.875 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj200 datafile 2023-12-01T06:37:32.042 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:32.042 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj201 datafile 2023-12-01T06:37:32.193 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:32.193 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj202 datafile 2023-12-01T06:37:32.335 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:32.335 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj203 datafile 2023-12-01T06:37:32.486 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:32.486 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj204 datafile 2023-12-01T06:37:32.628 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:32.629 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj205 datafile 2023-12-01T06:37:32.772 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:32.772 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj206 datafile 2023-12-01T06:37:32.930 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:32.930 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj207 datafile 2023-12-01T06:37:33.182 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:33.183 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj208 datafile 2023-12-01T06:37:33.350 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:33.350 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj209 datafile 2023-12-01T06:37:33.493 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:33.493 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj210 datafile 2023-12-01T06:37:33.643 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:33.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj211 datafile 2023-12-01T06:37:33.819 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:33.819 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj212 datafile 2023-12-01T06:37:33.979 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:33.979 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj213 datafile 2023-12-01T06:37:34.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:34.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj214 datafile 2023-12-01T06:37:34.306 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:34.306 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj215 datafile 2023-12-01T06:37:34.457 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:34.457 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj216 datafile 2023-12-01T06:37:34.609 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:34.609 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj217 datafile 2023-12-01T06:37:34.766 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:34.766 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj218 datafile 2023-12-01T06:37:34.926 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:34.926 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj219 datafile 2023-12-01T06:37:35.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:35.094 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj220 datafile 2023-12-01T06:37:35.236 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:35.236 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj221 datafile 2023-12-01T06:37:35.386 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:35.387 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj222 datafile 2023-12-01T06:37:35.546 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:35.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj223 datafile 2023-12-01T06:37:35.732 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:35.732 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj224 datafile 2023-12-01T06:37:35.924 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:35.925 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj225 datafile 2023-12-01T06:37:36.075 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:36.075 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj226 datafile 2023-12-01T06:37:36.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:36.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj227 datafile 2023-12-01T06:37:36.453 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:36.454 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj228 datafile 2023-12-01T06:37:36.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:36.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj229 datafile 2023-12-01T06:37:36.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:36.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj230 datafile 2023-12-01T06:37:36.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:36.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj231 datafile 2023-12-01T06:37:37.106 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:37.106 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj232 datafile 2023-12-01T06:37:37.271 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:37.271 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj233 datafile 2023-12-01T06:37:37.424 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:37.424 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj234 datafile 2023-12-01T06:37:37.575 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:37.575 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj235 datafile 2023-12-01T06:37:37.737 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:37.738 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj236 datafile 2023-12-01T06:37:37.910 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:37.911 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj237 datafile 2023-12-01T06:37:38.078 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:38.078 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj238 datafile 2023-12-01T06:37:38.496 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:38.496 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj239 datafile 2023-12-01T06:37:38.656 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:38.656 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj240 datafile 2023-12-01T06:37:38.815 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:38.816 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj241 datafile 2023-12-01T06:37:38.966 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:38.966 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj242 datafile 2023-12-01T06:37:39.125 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:39.125 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj243 datafile 2023-12-01T06:37:39.335 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:39.335 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj244 datafile 2023-12-01T06:37:39.502 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:39.502 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj245 datafile 2023-12-01T06:37:39.661 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:39.661 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj246 datafile 2023-12-01T06:37:39.846 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:39.846 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj247 datafile 2023-12-01T06:37:40.005 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:40.005 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj248 datafile 2023-12-01T06:37:40.165 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:40.165 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj249 datafile 2023-12-01T06:37:40.324 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:40.324 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj250 datafile 2023-12-01T06:37:40.466 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:40.466 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj251 datafile 2023-12-01T06:37:40.627 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:40.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj252 datafile 2023-12-01T06:37:40.785 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:40.785 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj253 datafile 2023-12-01T06:37:40.944 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:40.944 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj254 datafile 2023-12-01T06:37:41.095 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:41.095 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj255 datafile 2023-12-01T06:37:41.246 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:41.246 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj256 datafile 2023-12-01T06:37:41.405 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:41.405 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj257 datafile 2023-12-01T06:37:41.565 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:41.565 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj258 datafile 2023-12-01T06:37:41.765 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:41.765 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj259 datafile 2023-12-01T06:37:41.943 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:41.943 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj260 datafile 2023-12-01T06:37:42.101 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:42.101 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj261 datafile 2023-12-01T06:37:42.269 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:42.269 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj262 datafile 2023-12-01T06:37:42.427 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:42.428 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj263 datafile 2023-12-01T06:37:42.571 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:42.571 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj264 datafile 2023-12-01T06:37:42.738 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:42.738 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj265 datafile 2023-12-01T06:37:42.897 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:42.898 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj266 datafile 2023-12-01T06:37:43.065 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:43.065 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj267 datafile 2023-12-01T06:37:43.224 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:43.224 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj268 datafile 2023-12-01T06:37:43.393 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:43.393 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj269 datafile 2023-12-01T06:37:43.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:43.662 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj270 datafile 2023-12-01T06:37:43.861 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:43.861 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj271 datafile 2023-12-01T06:37:44.037 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:44.037 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj272 datafile 2023-12-01T06:37:44.205 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:44.205 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj273 datafile 2023-12-01T06:37:44.373 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:44.373 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj274 datafile 2023-12-01T06:37:44.523 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:44.523 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj275 datafile 2023-12-01T06:37:44.675 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:44.676 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj276 datafile 2023-12-01T06:37:44.833 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:44.833 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj277 datafile 2023-12-01T06:37:44.993 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:44.993 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj278 datafile 2023-12-01T06:37:45.151 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:45.151 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj279 datafile 2023-12-01T06:37:45.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:45.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj280 datafile 2023-12-01T06:37:45.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:45.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj281 datafile 2023-12-01T06:37:45.682 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:45.682 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj282 datafile 2023-12-01T06:37:45.866 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:45.866 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj283 datafile 2023-12-01T06:37:46.039 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:46.039 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj284 datafile 2023-12-01T06:37:46.183 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:46.183 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj285 datafile 2023-12-01T06:37:46.334 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:46.334 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj286 datafile 2023-12-01T06:37:46.494 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:46.494 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj287 datafile 2023-12-01T06:37:46.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:46.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj288 datafile 2023-12-01T06:37:46.804 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:46.804 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj289 datafile 2023-12-01T06:37:46.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:46.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj290 datafile 2023-12-01T06:37:47.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:47.148 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj291 datafile 2023-12-01T06:37:47.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:47.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj292 datafile 2023-12-01T06:37:47.508 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:47.508 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj293 datafile 2023-12-01T06:37:47.667 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:47.667 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj294 datafile 2023-12-01T06:37:47.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:47.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj295 datafile 2023-12-01T06:37:48.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:48.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj296 datafile 2023-12-01T06:37:48.178 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:48.178 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj297 datafile 2023-12-01T06:37:48.363 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:48.363 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj298 datafile 2023-12-01T06:37:48.513 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:48.513 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj299 datafile 2023-12-01T06:37:48.690 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:48.690 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj300 datafile 2023-12-01T06:37:48.867 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:48.867 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj301 datafile 2023-12-01T06:37:49.025 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:49.025 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj302 datafile 2023-12-01T06:37:49.168 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:49.168 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj303 datafile 2023-12-01T06:37:49.335 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:49.335 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj304 datafile 2023-12-01T06:37:49.486 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:49.486 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj305 datafile 2023-12-01T06:37:49.645 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:49.646 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj306 datafile 2023-12-01T06:37:49.838 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:49.838 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj307 datafile 2023-12-01T06:37:49.989 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:49.989 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj308 datafile 2023-12-01T06:37:50.156 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:50.156 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj309 datafile 2023-12-01T06:37:50.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:50.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj310 datafile 2023-12-01T06:37:50.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:50.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj311 datafile 2023-12-01T06:37:50.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:50.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj312 datafile 2023-12-01T06:37:50.777 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:50.777 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj313 datafile 2023-12-01T06:37:50.927 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:50.928 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj314 datafile 2023-12-01T06:37:51.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:51.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj315 datafile 2023-12-01T06:37:51.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:51.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj316 datafile 2023-12-01T06:37:51.430 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:51.430 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj317 datafile 2023-12-01T06:37:51.583 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:51.583 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj318 datafile 2023-12-01T06:37:51.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:51.759 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj319 datafile 2023-12-01T06:37:51.936 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:51.936 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj320 datafile 2023-12-01T06:37:52.094 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:52.094 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj321 datafile 2023-12-01T06:37:52.254 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:52.254 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj322 datafile 2023-12-01T06:37:52.412 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:52.413 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj323 datafile 2023-12-01T06:37:52.597 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:52.597 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj324 datafile 2023-12-01T06:37:52.747 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:52.748 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj325 datafile 2023-12-01T06:37:52.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:52.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj326 datafile 2023-12-01T06:37:53.082 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:53.083 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj327 datafile 2023-12-01T06:37:53.243 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:53.243 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj328 datafile 2023-12-01T06:37:53.393 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:53.394 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj329 datafile 2023-12-01T06:37:53.552 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:53.552 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj330 datafile 2023-12-01T06:37:53.711 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:53.711 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj331 datafile 2023-12-01T06:37:53.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:53.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj332 datafile 2023-12-01T06:37:54.039 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:54.039 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj333 datafile 2023-12-01T06:37:54.207 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:54.207 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj334 datafile 2023-12-01T06:37:54.366 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:54.366 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj335 datafile 2023-12-01T06:37:54.534 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:54.534 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj336 datafile 2023-12-01T06:37:54.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:54.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj337 datafile 2023-12-01T06:37:54.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:54.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj338 datafile 2023-12-01T06:37:54.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:54.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj339 datafile 2023-12-01T06:37:55.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:55.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj340 datafile 2023-12-01T06:37:55.355 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:55.355 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj341 datafile 2023-12-01T06:37:55.523 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:55.523 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj342 datafile 2023-12-01T06:37:55.690 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:55.691 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj343 datafile 2023-12-01T06:37:55.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:55.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj344 datafile 2023-12-01T06:37:56.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:56.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj345 datafile 2023-12-01T06:37:56.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:56.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj346 datafile 2023-12-01T06:37:56.327 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:56.328 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj347 datafile 2023-12-01T06:37:56.470 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:56.470 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj348 datafile 2023-12-01T06:37:56.630 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:56.630 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj349 datafile 2023-12-01T06:37:56.789 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:56.789 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj350 datafile 2023-12-01T06:37:56.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:56.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj351 datafile 2023-12-01T06:37:57.133 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:57.133 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj352 datafile 2023-12-01T06:37:57.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:57.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj353 datafile 2023-12-01T06:37:57.467 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:57.467 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj354 datafile 2023-12-01T06:37:57.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:57.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj355 datafile 2023-12-01T06:37:57.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:57.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj356 datafile 2023-12-01T06:37:57.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:57.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj357 datafile 2023-12-01T06:37:58.121 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:58.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj358 datafile 2023-12-01T06:37:58.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:58.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj359 datafile 2023-12-01T06:37:58.440 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:58.440 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj360 datafile 2023-12-01T06:37:58.591 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:58.591 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj361 datafile 2023-12-01T06:37:58.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:58.759 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj362 datafile 2023-12-01T06:37:58.939 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:58.939 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj363 datafile 2023-12-01T06:37:59.078 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:59.078 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj364 datafile 2023-12-01T06:37:59.237 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:59.237 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj365 datafile 2023-12-01T06:37:59.396 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:59.396 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj366 datafile 2023-12-01T06:37:59.555 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:59.555 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj367 datafile 2023-12-01T06:37:59.698 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:59.698 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj368 datafile 2023-12-01T06:37:59.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:37:59.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj369 datafile 2023-12-01T06:38:00.008 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:00.009 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj370 datafile 2023-12-01T06:38:00.159 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:00.159 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj371 datafile 2023-12-01T06:38:00.336 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:00.336 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj372 datafile 2023-12-01T06:38:00.486 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:00.486 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj373 datafile 2023-12-01T06:38:00.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:00.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj374 datafile 2023-12-01T06:38:00.788 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:00.788 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj375 datafile 2023-12-01T06:38:00.930 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:00.931 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj376 datafile 2023-12-01T06:38:01.098 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:01.098 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj377 datafile 2023-12-01T06:38:01.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:01.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj378 datafile 2023-12-01T06:38:01.425 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:01.425 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj379 datafile 2023-12-01T06:38:01.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:01.577 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj380 datafile 2023-12-01T06:38:01.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:01.744 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj381 datafile 2023-12-01T06:38:01.904 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:01.904 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj382 datafile 2023-12-01T06:38:02.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:02.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj383 datafile 2023-12-01T06:38:02.222 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:02.222 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj384 datafile 2023-12-01T06:38:02.373 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:02.373 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj385 datafile 2023-12-01T06:38:02.523 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:02.524 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj386 datafile 2023-12-01T06:38:02.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:02.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj387 datafile 2023-12-01T06:38:02.825 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:02.826 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj388 datafile 2023-12-01T06:38:02.985 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:02.985 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj389 datafile 2023-12-01T06:38:03.127 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:03.127 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj390 datafile 2023-12-01T06:38:03.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:03.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj391 datafile 2023-12-01T06:38:03.480 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:03.480 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj392 datafile 2023-12-01T06:38:03.641 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:03.641 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj393 datafile 2023-12-01T06:38:03.842 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:03.842 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj394 datafile 2023-12-01T06:38:04.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:04.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj395 datafile 2023-12-01T06:38:04.219 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:04.220 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj396 datafile 2023-12-01T06:38:04.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:04.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj397 datafile 2023-12-01T06:38:04.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:04.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj398 datafile 2023-12-01T06:38:04.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:04.655 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj399 datafile 2023-12-01T06:38:04.798 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:04.798 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj400 datafile 2023-12-01T06:38:04.940 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:04.940 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj401 datafile 2023-12-01T06:38:05.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:05.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj402 datafile 2023-12-01T06:38:05.242 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:05.242 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj403 datafile 2023-12-01T06:38:05.376 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:05.376 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj404 datafile 2023-12-01T06:38:05.536 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:05.537 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj405 datafile 2023-12-01T06:38:05.686 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:05.686 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj406 datafile 2023-12-01T06:38:05.846 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:05.846 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj407 datafile 2023-12-01T06:38:06.005 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:06.005 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj408 datafile 2023-12-01T06:38:06.148 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:06.148 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj409 datafile 2023-12-01T06:38:06.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:06.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj410 datafile 2023-12-01T06:38:06.483 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:06.483 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj411 datafile 2023-12-01T06:38:06.667 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:06.668 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj412 datafile 2023-12-01T06:38:06.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:06.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj413 datafile 2023-12-01T06:38:06.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:06.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj414 datafile 2023-12-01T06:38:07.129 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:07.129 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj415 datafile 2023-12-01T06:38:07.298 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:07.298 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj416 datafile 2023-12-01T06:38:07.447 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:07.447 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj417 datafile 2023-12-01T06:38:07.614 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:07.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj418 datafile 2023-12-01T06:38:07.799 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:07.799 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj419 datafile 2023-12-01T06:38:07.967 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:07.967 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj420 datafile 2023-12-01T06:38:08.127 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:08.127 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj421 datafile 2023-12-01T06:38:08.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:08.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj422 datafile 2023-12-01T06:38:08.436 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:08.436 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj423 datafile 2023-12-01T06:38:08.587 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:08.587 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj424 datafile 2023-12-01T06:38:08.739 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:08.739 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj425 datafile 2023-12-01T06:38:08.906 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:08.906 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj426 datafile 2023-12-01T06:38:09.056 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:09.057 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj427 datafile 2023-12-01T06:38:09.508 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:09.508 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj428 datafile 2023-12-01T06:38:09.659 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:09.659 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj429 datafile 2023-12-01T06:38:09.835 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:09.835 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj430 datafile 2023-12-01T06:38:09.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:09.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj431 datafile 2023-12-01T06:38:10.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:10.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj432 datafile 2023-12-01T06:38:10.305 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:10.305 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj433 datafile 2023-12-01T06:38:10.447 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:10.448 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj434 datafile 2023-12-01T06:38:10.598 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:10.598 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj435 datafile 2023-12-01T06:38:10.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:10.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj436 datafile 2023-12-01T06:38:10.917 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:10.917 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj437 datafile 2023-12-01T06:38:11.060 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:11.060 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj438 datafile 2023-12-01T06:38:11.227 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:11.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj439 datafile 2023-12-01T06:38:11.378 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:11.450 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj440 datafile 2023-12-01T06:38:11.529 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:11.529 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj441 datafile 2023-12-01T06:38:11.681 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:11.681 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj442 datafile 2023-12-01T06:38:11.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:11.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj443 datafile 2023-12-01T06:38:12.033 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:12.033 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj444 datafile 2023-12-01T06:38:12.183 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:12.183 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj445 datafile 2023-12-01T06:38:12.376 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:12.376 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj446 datafile 2023-12-01T06:38:12.528 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:12.528 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj447 datafile 2023-12-01T06:38:12.678 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:12.678 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj448 datafile 2023-12-01T06:38:12.854 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:12.854 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj449 datafile 2023-12-01T06:38:12.996 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:12.996 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj450 datafile 2023-12-01T06:38:13.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:13.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj451 datafile 2023-12-01T06:38:13.306 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:13.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj452 datafile 2023-12-01T06:38:13.458 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:13.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj453 datafile 2023-12-01T06:38:13.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:13.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj454 datafile 2023-12-01T06:38:13.767 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:13.768 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj455 datafile 2023-12-01T06:38:13.970 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:13.971 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj456 datafile 2023-12-01T06:38:14.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:14.128 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj457 datafile 2023-12-01T06:38:14.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:14.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj458 datafile 2023-12-01T06:38:14.438 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:14.438 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj459 datafile 2023-12-01T06:38:14.698 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:14.698 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj460 datafile 2023-12-01T06:38:14.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:14.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj461 datafile 2023-12-01T06:38:15.017 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:15.017 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj462 datafile 2023-12-01T06:38:15.168 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:15.168 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj463 datafile 2023-12-01T06:38:15.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:15.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj464 datafile 2023-12-01T06:38:15.511 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:15.511 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj465 datafile 2023-12-01T06:38:15.670 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:15.670 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj466 datafile 2023-12-01T06:38:15.862 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:15.862 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj467 datafile 2023-12-01T06:38:16.014 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:16.015 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj468 datafile 2023-12-01T06:38:16.182 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:16.182 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj469 datafile 2023-12-01T06:38:16.342 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:16.342 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj470 datafile 2023-12-01T06:38:16.492 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:16.493 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj471 datafile 2023-12-01T06:38:16.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:16.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj472 datafile 2023-12-01T06:38:16.794 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:16.795 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj473 datafile 2023-12-01T06:38:16.953 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:16.953 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj474 datafile 2023-12-01T06:38:17.096 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:17.096 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj475 datafile 2023-12-01T06:38:17.263 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:17.264 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj476 datafile 2023-12-01T06:38:17.416 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:17.416 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj477 datafile 2023-12-01T06:38:17.575 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:17.575 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj478 datafile 2023-12-01T06:38:17.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:17.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj479 datafile 2023-12-01T06:38:17.910 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:17.910 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj480 datafile 2023-12-01T06:38:18.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:18.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj481 datafile 2023-12-01T06:38:18.211 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:18.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj482 datafile 2023-12-01T06:38:18.387 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:18.387 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj483 datafile 2023-12-01T06:38:18.563 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:18.563 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj484 datafile 2023-12-01T06:38:18.706 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:18.706 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj485 datafile 2023-12-01T06:38:18.882 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:18.882 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj486 datafile 2023-12-01T06:38:19.050 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:19.050 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj487 datafile 2023-12-01T06:38:19.201 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:19.201 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj488 datafile 2023-12-01T06:38:19.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:19.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj489 datafile 2023-12-01T06:38:19.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:19.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj490 datafile 2023-12-01T06:38:19.669 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:19.669 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj491 datafile 2023-12-01T06:38:19.821 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:19.821 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj492 datafile 2023-12-01T06:38:19.972 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:19.972 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj493 datafile 2023-12-01T06:38:20.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:20.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj494 datafile 2023-12-01T06:38:20.266 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:20.266 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj495 datafile 2023-12-01T06:38:20.425 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:20.426 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj496 datafile 2023-12-01T06:38:20.584 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:20.585 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj497 datafile 2023-12-01T06:38:20.768 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:20.768 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj498 datafile 2023-12-01T06:38:20.921 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:20.921 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj499 datafile 2023-12-01T06:38:21.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:21.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj500 datafile 2023-12-01T06:38:21.229 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:21.229 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj501 datafile 2023-12-01T06:38:21.405 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:21.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj502 datafile 2023-12-01T06:38:21.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:21.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj503 datafile 2023-12-01T06:38:21.716 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:21.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj504 datafile 2023-12-01T06:38:21.876 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:21.876 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj505 datafile 2023-12-01T06:38:22.027 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:22.027 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj506 datafile 2023-12-01T06:38:22.178 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:22.178 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj507 datafile 2023-12-01T06:38:22.337 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:22.337 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj508 datafile 2023-12-01T06:38:22.488 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:22.488 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj509 datafile 2023-12-01T06:38:22.647 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:22.647 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj510 datafile 2023-12-01T06:38:22.797 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:22.798 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj511 datafile 2023-12-01T06:38:22.949 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:22.950 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj512 datafile 2023-12-01T06:38:23.108 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:23.109 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj513 datafile 2023-12-01T06:38:23.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:23.281 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj514 datafile 2023-12-01T06:38:23.435 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:23.435 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj515 datafile 2023-12-01T06:38:23.611 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:23.611 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj516 datafile 2023-12-01T06:38:23.763 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:23.763 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj517 datafile 2023-12-01T06:38:23.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:23.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj518 datafile 2023-12-01T06:38:24.115 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:24.115 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj519 datafile 2023-12-01T06:38:24.265 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:24.266 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj520 datafile 2023-12-01T06:38:24.441 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:24.441 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj521 datafile 2023-12-01T06:38:24.609 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:24.609 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj522 datafile 2023-12-01T06:38:24.769 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:24.769 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj523 datafile 2023-12-01T06:38:24.928 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:24.928 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj524 datafile 2023-12-01T06:38:25.087 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:25.087 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj525 datafile 2023-12-01T06:38:25.246 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:25.246 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj526 datafile 2023-12-01T06:38:25.405 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:25.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj527 datafile 2023-12-01T06:38:25.548 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:25.548 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj528 datafile 2023-12-01T06:38:25.699 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:25.699 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj529 datafile 2023-12-01T06:38:25.842 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:25.842 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj530 datafile 2023-12-01T06:38:25.993 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:25.993 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj531 datafile 2023-12-01T06:38:26.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:26.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj532 datafile 2023-12-01T06:38:26.336 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:26.336 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj533 datafile 2023-12-01T06:38:26.497 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:26.497 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj534 datafile 2023-12-01T06:38:26.639 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:26.639 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj535 datafile 2023-12-01T06:38:26.797 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:26.798 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj536 datafile 2023-12-01T06:38:26.957 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:26.957 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj537 datafile 2023-12-01T06:38:27.108 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:27.108 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj538 datafile 2023-12-01T06:38:27.259 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:27.259 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj539 datafile 2023-12-01T06:38:27.427 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:27.427 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj540 datafile 2023-12-01T06:38:27.586 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:27.586 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj541 datafile 2023-12-01T06:38:27.753 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:27.754 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj542 datafile 2023-12-01T06:38:27.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:27.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj543 datafile 2023-12-01T06:38:28.105 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:28.106 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj544 datafile 2023-12-01T06:38:28.257 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:28.257 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj545 datafile 2023-12-01T06:38:28.400 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:28.400 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj546 datafile 2023-12-01T06:38:28.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:28.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj547 datafile 2023-12-01T06:38:28.702 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:28.702 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj548 datafile 2023-12-01T06:38:28.852 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:28.852 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj549 datafile 2023-12-01T06:38:29.022 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:29.022 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj550 datafile 2023-12-01T06:38:29.206 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:29.206 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj551 datafile 2023-12-01T06:38:29.357 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:29.357 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj552 datafile 2023-12-01T06:38:29.514 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:29.515 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj553 datafile 2023-12-01T06:38:29.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:29.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj554 datafile 2023-12-01T06:38:29.852 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:29.852 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj555 datafile 2023-12-01T06:38:30.026 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:30.026 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj556 datafile 2023-12-01T06:38:30.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:30.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj557 datafile 2023-12-01T06:38:30.328 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:30.329 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj558 datafile 2023-12-01T06:38:30.513 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:30.513 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj559 datafile 2023-12-01T06:38:30.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:30.664 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj560 datafile 2023-12-01T06:38:30.830 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:30.831 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj561 datafile 2023-12-01T06:38:30.999 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:30.999 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj562 datafile 2023-12-01T06:38:31.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:31.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj563 datafile 2023-12-01T06:38:31.334 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:31.334 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj564 datafile 2023-12-01T06:38:31.493 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:31.493 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj565 datafile 2023-12-01T06:38:31.636 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:31.636 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj566 datafile 2023-12-01T06:38:31.803 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:31.804 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj567 datafile 2023-12-01T06:38:31.980 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:31.980 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj568 datafile 2023-12-01T06:38:32.139 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:32.139 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj569 datafile 2023-12-01T06:38:32.289 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:32.290 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj570 datafile 2023-12-01T06:38:32.440 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:32.441 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj571 datafile 2023-12-01T06:38:32.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:32.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj572 datafile 2023-12-01T06:38:32.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:32.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj573 datafile 2023-12-01T06:38:32.885 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:32.885 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj574 datafile 2023-12-01T06:38:33.035 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:33.036 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj575 datafile 2023-12-01T06:38:33.203 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:33.203 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj576 datafile 2023-12-01T06:38:33.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:33.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj577 datafile 2023-12-01T06:38:33.514 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:33.514 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj578 datafile 2023-12-01T06:38:33.664 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:33.664 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj579 datafile 2023-12-01T06:38:33.907 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:33.907 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj580 datafile 2023-12-01T06:38:34.080 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:34.080 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj581 datafile 2023-12-01T06:38:34.243 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:34.243 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj582 datafile 2023-12-01T06:38:34.385 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:34.386 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj583 datafile 2023-12-01T06:38:34.537 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:34.537 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj584 datafile 2023-12-01T06:38:34.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:34.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj585 datafile 2023-12-01T06:38:34.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:34.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj586 datafile 2023-12-01T06:38:35.145 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:35.145 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj587 datafile 2023-12-01T06:38:35.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:35.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj588 datafile 2023-12-01T06:38:35.483 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:35.483 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj589 datafile 2023-12-01T06:38:35.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:35.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj590 datafile 2023-12-01T06:38:35.768 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:35.768 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj591 datafile 2023-12-01T06:38:35.953 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:35.953 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj592 datafile 2023-12-01T06:38:36.112 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:36.113 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj593 datafile 2023-12-01T06:38:36.280 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:36.280 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj594 datafile 2023-12-01T06:38:36.466 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:36.466 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj595 datafile 2023-12-01T06:38:36.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:36.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj596 datafile 2023-12-01T06:38:36.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:36.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj597 datafile 2023-12-01T06:38:36.942 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:36.942 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj598 datafile 2023-12-01T06:38:37.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:37.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj599 datafile 2023-12-01T06:38:37.252 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:37.253 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj600 datafile 2023-12-01T06:38:37.411 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:37.411 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj601 datafile 2023-12-01T06:38:37.554 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:37.554 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj602 datafile 2023-12-01T06:38:37.705 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:37.705 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj603 datafile 2023-12-01T06:38:37.864 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:37.864 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj604 datafile 2023-12-01T06:38:37.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:37.999 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj605 datafile 2023-12-01T06:38:38.150 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:38.150 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj606 datafile 2023-12-01T06:38:38.301 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:38.301 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj607 datafile 2023-12-01T06:38:38.451 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:38.451 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj608 datafile 2023-12-01T06:38:38.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:38.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj609 datafile 2023-12-01T06:38:38.744 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:38.745 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj610 datafile 2023-12-01T06:38:38.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:38.887 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj611 datafile 2023-12-01T06:38:39.038 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:39.038 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj612 datafile 2023-12-01T06:38:39.206 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:39.207 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj613 datafile 2023-12-01T06:38:39.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:39.349 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj614 datafile 2023-12-01T06:38:39.508 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:39.508 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj615 datafile 2023-12-01T06:38:39.651 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:39.651 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj616 datafile 2023-12-01T06:38:39.845 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:39.845 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj617 datafile 2023-12-01T06:38:40.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:40.021 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj618 datafile 2023-12-01T06:38:40.454 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:40.455 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj619 datafile 2023-12-01T06:38:40.597 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:40.597 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj620 datafile 2023-12-01T06:38:40.757 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:40.757 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj621 datafile 2023-12-01T06:38:40.907 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:40.907 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj622 datafile 2023-12-01T06:38:41.074 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:41.075 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj623 datafile 2023-12-01T06:38:41.242 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:41.243 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj624 datafile 2023-12-01T06:38:41.393 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:41.393 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj625 datafile 2023-12-01T06:38:41.545 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:41.545 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj626 datafile 2023-12-01T06:38:41.704 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:41.704 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj627 datafile 2023-12-01T06:38:41.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:41.879 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj628 datafile 2023-12-01T06:38:42.039 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:42.066 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj629 datafile 2023-12-01T06:38:42.198 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:42.198 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj630 datafile 2023-12-01T06:38:42.357 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:42.357 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj631 datafile 2023-12-01T06:38:42.534 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:42.534 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj632 datafile 2023-12-01T06:38:42.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:42.685 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj633 datafile 2023-12-01T06:38:42.836 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:42.836 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj634 datafile 2023-12-01T06:38:42.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:42.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj635 datafile 2023-12-01T06:38:43.154 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:43.154 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj636 datafile 2023-12-01T06:38:43.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:43.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj637 datafile 2023-12-01T06:38:43.489 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:43.489 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj638 datafile 2023-12-01T06:38:43.648 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:43.648 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj639 datafile 2023-12-01T06:38:43.816 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:43.816 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj640 datafile 2023-12-01T06:38:43.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:43.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj641 datafile 2023-12-01T06:38:44.126 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:44.126 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj642 datafile 2023-12-01T06:38:44.295 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:44.295 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj643 datafile 2023-12-01T06:38:44.446 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:44.446 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj644 datafile 2023-12-01T06:38:44.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:44.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj645 datafile 2023-12-01T06:38:44.755 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:44.755 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj646 datafile 2023-12-01T06:38:44.889 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:44.889 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj647 datafile 2023-12-01T06:38:45.040 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:45.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj648 datafile 2023-12-01T06:38:45.208 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:45.208 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj649 datafile 2023-12-01T06:38:45.359 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:45.359 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj650 datafile 2023-12-01T06:38:45.660 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:45.660 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj651 datafile 2023-12-01T06:38:45.803 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:45.803 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj652 datafile 2023-12-01T06:38:45.971 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:45.971 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj653 datafile 2023-12-01T06:38:46.139 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:46.139 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj654 datafile 2023-12-01T06:38:46.289 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:46.290 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj655 datafile 2023-12-01T06:38:46.441 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:46.441 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj656 datafile 2023-12-01T06:38:46.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:46.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj657 datafile 2023-12-01T06:38:46.744 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:46.744 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj658 datafile 2023-12-01T06:38:46.895 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:46.895 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj659 datafile 2023-12-01T06:38:47.062 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:47.062 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj660 datafile 2023-12-01T06:38:47.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:47.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj661 datafile 2023-12-01T06:38:47.381 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:47.382 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj662 datafile 2023-12-01T06:38:47.540 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:47.541 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj663 datafile 2023-12-01T06:38:47.692 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:47.692 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj664 datafile 2023-12-01T06:38:47.861 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:47.862 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj665 datafile 2023-12-01T06:38:48.003 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:48.003 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj666 datafile 2023-12-01T06:38:48.163 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:48.163 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj667 datafile 2023-12-01T06:38:48.329 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:48.329 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj668 datafile 2023-12-01T06:38:48.522 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:48.522 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj669 datafile 2023-12-01T06:38:48.672 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:48.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj670 datafile 2023-12-01T06:38:48.853 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:48.854 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj671 datafile 2023-12-01T06:38:49.008 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:49.008 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj672 datafile 2023-12-01T06:38:49.168 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:49.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj673 datafile 2023-12-01T06:38:49.327 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:49.327 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj674 datafile 2023-12-01T06:38:49.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:49.478 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj675 datafile 2023-12-01T06:38:49.628 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:49.628 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj676 datafile 2023-12-01T06:38:49.802 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:49.802 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj677 datafile 2023-12-01T06:38:49.981 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:49.982 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj678 datafile 2023-12-01T06:38:50.149 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:50.149 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj679 datafile 2023-12-01T06:38:50.282 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:50.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj680 datafile 2023-12-01T06:38:50.425 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:50.425 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj681 datafile 2023-12-01T06:38:50.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:50.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj682 datafile 2023-12-01T06:38:50.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:50.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj683 datafile 2023-12-01T06:38:50.895 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:50.895 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj684 datafile 2023-12-01T06:38:51.046 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:51.046 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj685 datafile 2023-12-01T06:38:51.205 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:51.205 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj686 datafile 2023-12-01T06:38:51.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:51.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj687 datafile 2023-12-01T06:38:51.532 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:51.532 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj688 datafile 2023-12-01T06:38:51.716 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:51.716 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj689 datafile 2023-12-01T06:38:51.885 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:51.885 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj690 datafile 2023-12-01T06:38:52.027 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:52.027 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj691 datafile 2023-12-01T06:38:52.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:52.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj692 datafile 2023-12-01T06:38:52.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:52.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj693 datafile 2023-12-01T06:38:52.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:52.513 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj694 datafile 2023-12-01T06:38:52.682 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:52.682 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj695 datafile 2023-12-01T06:38:52.832 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:52.832 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj696 datafile 2023-12-01T06:38:52.992 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:52.992 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj697 datafile 2023-12-01T06:38:53.150 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:53.150 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj698 datafile 2023-12-01T06:38:53.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:53.318 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj699 datafile 2023-12-01T06:38:53.469 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:53.469 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj700 datafile 2023-12-01T06:38:53.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:53.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj701 datafile 2023-12-01T06:38:53.786 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:53.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj702 datafile 2023-12-01T06:38:53.973 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:53.973 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj703 datafile 2023-12-01T06:38:54.114 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:54.114 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj704 datafile 2023-12-01T06:38:54.274 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:54.274 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj705 datafile 2023-12-01T06:38:54.426 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:54.426 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj706 datafile 2023-12-01T06:38:54.593 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:54.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj707 datafile 2023-12-01T06:38:54.751 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:54.752 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj708 datafile 2023-12-01T06:38:54.902 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:54.902 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj709 datafile 2023-12-01T06:38:55.054 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:55.054 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj710 datafile 2023-12-01T06:38:55.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:55.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj711 datafile 2023-12-01T06:38:55.381 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:55.381 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj712 datafile 2023-12-01T06:38:55.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:55.540 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj713 datafile 2023-12-01T06:38:55.699 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:55.699 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj714 datafile 2023-12-01T06:38:55.867 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:55.867 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj715 datafile 2023-12-01T06:38:56.035 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:56.035 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj716 datafile 2023-12-01T06:38:56.193 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:56.193 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj717 datafile 2023-12-01T06:38:56.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:56.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj718 datafile 2023-12-01T06:38:56.520 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:56.520 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj719 datafile 2023-12-01T06:38:56.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:56.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj720 datafile 2023-12-01T06:38:56.848 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:56.848 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj721 datafile 2023-12-01T06:38:56.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:56.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj722 datafile 2023-12-01T06:38:57.149 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:57.149 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj723 datafile 2023-12-01T06:38:57.333 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:57.333 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj724 datafile 2023-12-01T06:38:57.492 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:57.492 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj725 datafile 2023-12-01T06:38:57.660 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:57.660 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj726 datafile 2023-12-01T06:38:57.828 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:57.828 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj727 datafile 2023-12-01T06:38:58.013 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:58.014 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj728 datafile 2023-12-01T06:38:58.189 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:58.189 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj729 datafile 2023-12-01T06:38:58.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:58.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj730 datafile 2023-12-01T06:38:58.499 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:58.500 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj731 datafile 2023-12-01T06:38:58.642 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:58.642 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj732 datafile 2023-12-01T06:38:58.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:58.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj733 datafile 2023-12-01T06:38:58.934 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:58.934 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj734 datafile 2023-12-01T06:38:59.094 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:59.094 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj735 datafile 2023-12-01T06:38:59.263 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:59.263 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj736 datafile 2023-12-01T06:38:59.421 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:59.421 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj737 datafile 2023-12-01T06:38:59.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:59.589 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj738 datafile 2023-12-01T06:38:59.756 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:59.757 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj739 datafile 2023-12-01T06:38:59.942 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:38:59.942 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj740 datafile 2023-12-01T06:39:00.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:00.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj741 datafile 2023-12-01T06:39:00.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:00.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj742 datafile 2023-12-01T06:39:00.435 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:00.435 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj743 datafile 2023-12-01T06:39:00.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:00.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj744 datafile 2023-12-01T06:39:00.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:00.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj745 datafile 2023-12-01T06:39:00.938 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:00.938 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj746 datafile 2023-12-01T06:39:01.097 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:01.097 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj747 datafile 2023-12-01T06:39:01.257 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:01.257 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj748 datafile 2023-12-01T06:39:01.416 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:01.416 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj749 datafile 2023-12-01T06:39:01.558 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:01.558 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj750 datafile 2023-12-01T06:39:01.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:01.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj751 datafile 2023-12-01T06:39:01.869 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:01.869 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj752 datafile 2023-12-01T06:39:02.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:02.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj753 datafile 2023-12-01T06:39:02.162 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:02.162 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj754 datafile 2023-12-01T06:39:02.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:02.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj755 datafile 2023-12-01T06:39:02.489 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:02.489 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj756 datafile 2023-12-01T06:39:02.640 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:02.640 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj757 datafile 2023-12-01T06:39:02.791 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:02.791 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj758 datafile 2023-12-01T06:39:02.967 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:02.967 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj759 datafile 2023-12-01T06:39:03.118 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:03.118 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj760 datafile 2023-12-01T06:39:03.269 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:03.269 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj761 datafile 2023-12-01T06:39:03.419 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:03.420 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj762 datafile 2023-12-01T06:39:03.609 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:03.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj763 datafile 2023-12-01T06:39:03.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:03.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj764 datafile 2023-12-01T06:39:03.990 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:03.990 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj765 datafile 2023-12-01T06:39:04.158 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:04.158 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj766 datafile 2023-12-01T06:39:04.325 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:04.325 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj767 datafile 2023-12-01T06:39:04.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:04.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj768 datafile 2023-12-01T06:39:04.636 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:04.636 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj769 datafile 2023-12-01T06:39:04.786 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:04.786 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj770 datafile 2023-12-01T06:39:04.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:04.945 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj771 datafile 2023-12-01T06:39:05.113 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:05.113 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj772 datafile 2023-12-01T06:39:05.272 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:05.273 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj773 datafile 2023-12-01T06:39:05.440 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:05.440 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj774 datafile 2023-12-01T06:39:05.582 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:05.582 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj775 datafile 2023-12-01T06:39:05.767 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:05.767 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj776 datafile 2023-12-01T06:39:05.960 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:05.960 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj777 datafile 2023-12-01T06:39:06.178 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:06.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj778 datafile 2023-12-01T06:39:06.337 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:06.337 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj779 datafile 2023-12-01T06:39:06.504 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:06.504 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj780 datafile 2023-12-01T06:39:06.664 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:06.664 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj781 datafile 2023-12-01T06:39:06.823 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:06.823 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj782 datafile 2023-12-01T06:39:06.974 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:06.974 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj783 datafile 2023-12-01T06:39:07.126 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:07.126 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj784 datafile 2023-12-01T06:39:07.310 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:07.310 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj785 datafile 2023-12-01T06:39:07.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:07.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj786 datafile 2023-12-01T06:39:07.611 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:07.611 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj787 datafile 2023-12-01T06:39:07.778 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:07.778 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj788 datafile 2023-12-01T06:39:07.964 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:07.964 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj789 datafile 2023-12-01T06:39:08.114 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:08.115 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj790 datafile 2023-12-01T06:39:08.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:08.283 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj791 datafile 2023-12-01T06:39:08.441 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:08.442 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj792 datafile 2023-12-01T06:39:08.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:08.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj793 datafile 2023-12-01T06:39:08.760 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:08.760 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj794 datafile 2023-12-01T06:39:08.911 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:08.911 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj795 datafile 2023-12-01T06:39:09.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:09.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj796 datafile 2023-12-01T06:39:09.247 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:09.247 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj797 datafile 2023-12-01T06:39:09.422 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:09.422 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj798 datafile 2023-12-01T06:39:09.598 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:09.598 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj799 datafile 2023-12-01T06:39:09.766 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:09.766 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj800 datafile 2023-12-01T06:39:09.960 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:09.960 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj801 datafile 2023-12-01T06:39:10.126 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:10.126 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj802 datafile 2023-12-01T06:39:10.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:10.295 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj803 datafile 2023-12-01T06:39:10.445 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:10.445 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj804 datafile 2023-12-01T06:39:10.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:10.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj805 datafile 2023-12-01T06:39:10.755 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:10.755 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj806 datafile 2023-12-01T06:39:10.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:10.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj807 datafile 2023-12-01T06:39:11.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:11.091 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj808 datafile 2023-12-01T06:39:11.449 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:11.449 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj809 datafile 2023-12-01T06:39:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:11.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj810 datafile 2023-12-01T06:39:11.760 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:11.760 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj811 datafile 2023-12-01T06:39:11.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:11.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj812 datafile 2023-12-01T06:39:12.104 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:12.104 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj813 datafile 2023-12-01T06:39:12.263 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:12.264 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj814 datafile 2023-12-01T06:39:12.439 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:12.439 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj815 datafile 2023-12-01T06:39:12.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:12.615 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj816 datafile 2023-12-01T06:39:12.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:12.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj817 datafile 2023-12-01T06:39:12.909 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:12.909 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj818 datafile 2023-12-01T06:39:13.059 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:13.059 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj819 datafile 2023-12-01T06:39:13.218 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:13.219 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj820 datafile 2023-12-01T06:39:13.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:13.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj821 datafile 2023-12-01T06:39:13.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:13.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj822 datafile 2023-12-01T06:39:13.664 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:13.664 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj823 datafile 2023-12-01T06:39:13.823 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:13.823 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj824 datafile 2023-12-01T06:39:13.982 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:13.982 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj825 datafile 2023-12-01T06:39:14.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:14.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj826 datafile 2023-12-01T06:39:14.300 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:14.301 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj827 datafile 2023-12-01T06:39:14.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:14.477 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj828 datafile 2023-12-01T06:39:14.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:14.620 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj829 datafile 2023-12-01T06:39:14.778 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:14.778 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj830 datafile 2023-12-01T06:39:14.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:14.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj831 datafile 2023-12-01T06:39:15.097 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:15.097 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj832 datafile 2023-12-01T06:39:15.264 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:15.264 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj833 datafile 2023-12-01T06:39:15.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:15.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj834 datafile 2023-12-01T06:39:15.575 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:15.575 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj835 datafile 2023-12-01T06:39:15.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:15.751 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj836 datafile 2023-12-01T06:39:15.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:15.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj837 datafile 2023-12-01T06:39:16.103 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:16.103 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj838 datafile 2023-12-01T06:39:16.398 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:16.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj839 datafile 2023-12-01T06:39:16.563 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:16.563 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj840 datafile 2023-12-01T06:39:16.706 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:16.707 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj841 datafile 2023-12-01T06:39:16.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:16.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj842 datafile 2023-12-01T06:39:17.050 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:17.050 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj843 datafile 2023-12-01T06:39:17.201 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:17.201 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj844 datafile 2023-12-01T06:39:17.360 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:17.360 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj845 datafile 2023-12-01T06:39:17.536 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:17.537 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj846 datafile 2023-12-01T06:39:17.687 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:17.687 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj847 datafile 2023-12-01T06:39:17.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:17.888 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj848 datafile 2023-12-01T06:39:18.056 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:18.056 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj849 datafile 2023-12-01T06:39:18.215 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:18.216 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj850 datafile 2023-12-01T06:39:18.350 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:18.350 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj851 datafile 2023-12-01T06:39:18.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:18.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj852 datafile 2023-12-01T06:39:18.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:18.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj853 datafile 2023-12-01T06:39:18.844 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:18.844 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj854 datafile 2023-12-01T06:39:19.012 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:19.012 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj855 datafile 2023-12-01T06:39:19.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:19.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj856 datafile 2023-12-01T06:39:19.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:19.390 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj857 datafile 2023-12-01T06:39:19.556 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:19.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj858 datafile 2023-12-01T06:39:19.707 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:19.707 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj859 datafile 2023-12-01T06:39:19.892 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:19.892 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj860 datafile 2023-12-01T06:39:20.060 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:20.061 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj861 datafile 2023-12-01T06:39:20.227 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:20.227 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj862 datafile 2023-12-01T06:39:20.386 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:20.386 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj863 datafile 2023-12-01T06:39:20.537 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:20.537 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj864 datafile 2023-12-01T06:39:20.697 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:20.697 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj865 datafile 2023-12-01T06:39:20.864 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:20.864 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj866 datafile 2023-12-01T06:39:21.024 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:21.024 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj867 datafile 2023-12-01T06:39:21.182 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:21.182 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj868 datafile 2023-12-01T06:39:21.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:21.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj869 datafile 2023-12-01T06:39:21.560 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:21.560 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj870 datafile 2023-12-01T06:39:21.727 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:21.727 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj871 datafile 2023-12-01T06:39:21.903 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:21.903 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj872 datafile 2023-12-01T06:39:22.054 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:22.054 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj873 datafile 2023-12-01T06:39:22.205 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:22.205 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj874 datafile 2023-12-01T06:39:22.374 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:22.374 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj875 datafile 2023-12-01T06:39:22.524 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:22.524 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj876 datafile 2023-12-01T06:39:22.700 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:22.700 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj877 datafile 2023-12-01T06:39:22.868 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:22.868 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj878 datafile 2023-12-01T06:39:23.027 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:23.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj879 datafile 2023-12-01T06:39:23.178 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:23.178 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj880 datafile 2023-12-01T06:39:23.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:23.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj881 datafile 2023-12-01T06:39:23.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:23.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj882 datafile 2023-12-01T06:39:23.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:23.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj883 datafile 2023-12-01T06:39:23.815 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:23.815 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj884 datafile 2023-12-01T06:39:23.973 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:23.974 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj885 datafile 2023-12-01T06:39:24.125 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:24.126 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj886 datafile 2023-12-01T06:39:24.293 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:24.293 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj887 datafile 2023-12-01T06:39:24.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:24.465 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj888 datafile 2023-12-01T06:39:24.636 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:24.636 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj889 datafile 2023-12-01T06:39:24.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:24.788 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj890 datafile 2023-12-01T06:39:24.930 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:24.930 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj891 datafile 2023-12-01T06:39:25.081 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:25.081 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj892 datafile 2023-12-01T06:39:25.232 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:25.232 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj893 datafile 2023-12-01T06:39:25.383 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:25.383 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj894 datafile 2023-12-01T06:39:25.560 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:25.560 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj895 datafile 2023-12-01T06:39:25.718 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:25.718 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj896 datafile 2023-12-01T06:39:25.911 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:25.911 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj897 datafile 2023-12-01T06:39:26.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:26.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj898 datafile 2023-12-01T06:39:26.229 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:26.230 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj899 datafile 2023-12-01T06:39:26.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:26.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj900 datafile 2023-12-01T06:39:26.540 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:26.540 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj901 datafile 2023-12-01T06:39:26.698 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:26.698 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj902 datafile 2023-12-01T06:39:26.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:26.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj903 datafile 2023-12-01T06:39:27.009 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:27.009 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj904 datafile 2023-12-01T06:39:27.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:27.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj905 datafile 2023-12-01T06:39:27.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:27.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj906 datafile 2023-12-01T06:39:27.504 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:27.504 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj907 datafile 2023-12-01T06:39:27.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:27.680 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj908 datafile 2023-12-01T06:39:27.856 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:27.856 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj909 datafile 2023-12-01T06:39:28.007 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:28.008 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj910 datafile 2023-12-01T06:39:28.157 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:28.157 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj911 datafile 2023-12-01T06:39:28.325 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:28.326 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj912 datafile 2023-12-01T06:39:28.484 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:28.485 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj913 datafile 2023-12-01T06:39:28.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:28.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj914 datafile 2023-12-01T06:39:28.803 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:28.803 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj915 datafile 2023-12-01T06:39:28.953 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:28.954 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj916 datafile 2023-12-01T06:39:29.096 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:29.097 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj917 datafile 2023-12-01T06:39:29.247 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:29.248 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj918 datafile 2023-12-01T06:39:29.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:29.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj919 datafile 2023-12-01T06:39:29.573 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:29.573 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj920 datafile 2023-12-01T06:39:29.733 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:29.733 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj921 datafile 2023-12-01T06:39:29.919 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:29.919 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj922 datafile 2023-12-01T06:39:30.086 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:30.086 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj923 datafile 2023-12-01T06:39:30.246 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:30.246 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj924 datafile 2023-12-01T06:39:30.405 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:30.405 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj925 datafile 2023-12-01T06:39:30.583 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:30.583 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj926 datafile 2023-12-01T06:39:30.751 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:30.751 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj927 datafile 2023-12-01T06:39:30.909 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:30.910 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj928 datafile 2023-12-01T06:39:31.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:31.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj929 datafile 2023-12-01T06:39:31.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:31.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj930 datafile 2023-12-01T06:39:31.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:31.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj931 datafile 2023-12-01T06:39:31.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:31.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj932 datafile 2023-12-01T06:39:31.714 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:31.714 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj933 datafile 2023-12-01T06:39:31.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:31.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj934 datafile 2023-12-01T06:39:32.033 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:32.033 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj935 datafile 2023-12-01T06:39:32.201 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:32.201 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj936 datafile 2023-12-01T06:39:32.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:32.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj937 datafile 2023-12-01T06:39:32.520 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:32.520 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj938 datafile 2023-12-01T06:39:32.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:32.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj939 datafile 2023-12-01T06:39:32.829 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:32.829 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj940 datafile 2023-12-01T06:39:32.973 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:32.973 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj941 datafile 2023-12-01T06:39:33.115 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:33.115 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj942 datafile 2023-12-01T06:39:33.282 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:33.282 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj943 datafile 2023-12-01T06:39:33.458 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:33.458 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj944 datafile 2023-12-01T06:39:33.634 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:33.634 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj945 datafile 2023-12-01T06:39:33.810 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:33.810 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj946 datafile 2023-12-01T06:39:34.046 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:34.046 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj947 datafile 2023-12-01T06:39:34.230 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:34.230 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj948 datafile 2023-12-01T06:39:34.397 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:34.397 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj949 datafile 2023-12-01T06:39:34.548 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:34.548 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj950 datafile 2023-12-01T06:39:34.699 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:34.699 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj951 datafile 2023-12-01T06:39:34.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:34.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj952 datafile 2023-12-01T06:39:35.010 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:35.010 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj953 datafile 2023-12-01T06:39:35.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:35.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj954 datafile 2023-12-01T06:39:35.337 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:35.337 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj955 datafile 2023-12-01T06:39:35.487 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:35.487 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj956 datafile 2023-12-01T06:39:35.638 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:35.639 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj957 datafile 2023-12-01T06:39:35.805 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:35.806 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj958 datafile 2023-12-01T06:39:35.983 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:35.983 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj959 datafile 2023-12-01T06:39:36.149 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:36.149 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj960 datafile 2023-12-01T06:39:36.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:36.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj961 datafile 2023-12-01T06:39:36.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:36.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj962 datafile 2023-12-01T06:39:36.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:36.645 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj963 datafile 2023-12-01T06:39:36.912 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:36.912 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj964 datafile 2023-12-01T06:39:37.080 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:37.080 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj965 datafile 2023-12-01T06:39:37.246 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:37.247 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj966 datafile 2023-12-01T06:39:37.398 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:37.398 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj967 datafile 2023-12-01T06:39:37.567 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:37.567 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj968 datafile 2023-12-01T06:39:37.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:37.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj969 datafile 2023-12-01T06:39:37.902 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:37.902 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj970 datafile 2023-12-01T06:39:38.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:38.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj971 datafile 2023-12-01T06:39:38.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:38.213 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj972 datafile 2023-12-01T06:39:38.363 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:38.363 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj973 datafile 2023-12-01T06:39:38.522 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:38.522 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj974 datafile 2023-12-01T06:39:38.672 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:38.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj975 datafile 2023-12-01T06:39:38.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:38.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj976 datafile 2023-12-01T06:39:39.017 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:39.017 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj977 datafile 2023-12-01T06:39:39.193 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:39.193 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj978 datafile 2023-12-01T06:39:39.360 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:39.360 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj979 datafile 2023-12-01T06:39:39.544 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:39.545 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj980 datafile 2023-12-01T06:39:39.721 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:39.722 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj981 datafile 2023-12-01T06:39:39.872 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:39.872 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj982 datafile 2023-12-01T06:39:40.040 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:40.040 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj983 datafile 2023-12-01T06:39:40.191 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:40.191 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj984 datafile 2023-12-01T06:39:40.350 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:40.350 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj985 datafile 2023-12-01T06:39:40.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:40.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj986 datafile 2023-12-01T06:39:40.652 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:40.652 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj987 datafile 2023-12-01T06:39:40.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:40.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj988 datafile 2023-12-01T06:39:40.962 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:40.962 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj989 datafile 2023-12-01T06:39:41.121 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:41.121 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj990 datafile 2023-12-01T06:39:41.280 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:41.280 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj991 datafile 2023-12-01T06:39:41.431 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:41.431 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj992 datafile 2023-12-01T06:39:41.607 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:41.607 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj993 datafile 2023-12-01T06:39:41.775 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:41.775 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj994 datafile 2023-12-01T06:39:42.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:42.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj995 datafile 2023-12-01T06:39:42.410 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:42.411 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj996 datafile 2023-12-01T06:39:42.578 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:42.579 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj997 datafile 2023-12-01T06:39:42.729 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:42.729 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj998 datafile 2023-12-01T06:39:42.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:42.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj999 datafile 2023-12-01T06:39:43.031 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:43.032 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test2 put obj1000 datafile 2023-12-01T06:39:43.192 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:74: TEST_recover_unexpected: for i in $(seq 1 $POOLS) 2023-12-01T06:39:43.192 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: seq 1 1000 2023-12-01T06:39:43.194 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:43.194 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj1 datafile 2023-12-01T06:39:43.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:43.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj2 datafile 2023-12-01T06:39:43.543 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:43.543 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj3 datafile 2023-12-01T06:39:43.702 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:43.702 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj4 datafile 2023-12-01T06:39:43.863 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:43.863 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj5 datafile 2023-12-01T06:39:44.029 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:44.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj6 datafile 2023-12-01T06:39:44.181 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:44.182 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj7 datafile 2023-12-01T06:39:44.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:44.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj8 datafile 2023-12-01T06:39:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:44.490 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj9 datafile 2023-12-01T06:39:44.659 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:44.659 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj10 datafile 2023-12-01T06:39:44.826 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:44.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj11 datafile 2023-12-01T06:39:45.002 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:45.002 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj12 datafile 2023-12-01T06:39:45.183 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:45.183 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj13 datafile 2023-12-01T06:39:45.370 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:45.370 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj14 datafile 2023-12-01T06:39:45.513 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:45.513 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj15 datafile 2023-12-01T06:39:45.690 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:45.690 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj16 datafile 2023-12-01T06:39:45.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:45.875 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj17 datafile 2023-12-01T06:39:46.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:46.042 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj18 datafile 2023-12-01T06:39:46.201 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:46.201 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj19 datafile 2023-12-01T06:39:46.352 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:46.352 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj20 datafile 2023-12-01T06:39:46.502 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:46.503 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj21 datafile 2023-12-01T06:39:46.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:46.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj22 datafile 2023-12-01T06:39:46.812 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:46.812 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj23 datafile 2023-12-01T06:39:46.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:46.956 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj24 datafile 2023-12-01T06:39:47.114 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:47.114 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj25 datafile 2023-12-01T06:39:47.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:47.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj26 datafile 2023-12-01T06:39:47.550 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:47.550 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj27 datafile 2023-12-01T06:39:47.709 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:47.709 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj28 datafile 2023-12-01T06:39:47.877 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:47.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj29 datafile 2023-12-01T06:39:48.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:48.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj30 datafile 2023-12-01T06:39:48.221 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:48.221 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj31 datafile 2023-12-01T06:39:48.380 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:48.381 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj32 datafile 2023-12-01T06:39:48.555 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:48.556 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj33 datafile 2023-12-01T06:39:48.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:48.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj34 datafile 2023-12-01T06:39:48.900 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:48.900 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj35 datafile 2023-12-01T06:39:49.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:49.093 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj36 datafile 2023-12-01T06:39:49.261 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:49.261 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj37 datafile 2023-12-01T06:39:49.415 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:49.416 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj38 datafile 2023-12-01T06:39:49.570 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:49.570 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj39 datafile 2023-12-01T06:39:49.720 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:49.720 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj40 datafile 2023-12-01T06:39:49.889 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:49.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj41 datafile 2023-12-01T06:39:50.048 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:50.050 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj42 datafile 2023-12-01T06:39:50.200 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:50.200 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj43 datafile 2023-12-01T06:39:50.358 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:50.359 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj44 datafile 2023-12-01T06:39:50.526 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:50.527 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj45 datafile 2023-12-01T06:39:50.694 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:50.694 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj46 datafile 2023-12-01T06:39:50.870 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:50.870 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj47 datafile 2023-12-01T06:39:51.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:51.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj48 datafile 2023-12-01T06:39:51.180 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:51.180 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj49 datafile 2023-12-01T06:39:51.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:51.331 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj50 datafile 2023-12-01T06:39:51.482 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:51.482 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj51 datafile 2023-12-01T06:39:51.658 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:51.658 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj52 datafile 2023-12-01T06:39:51.826 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:51.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj53 datafile 2023-12-01T06:39:52.011 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:52.011 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj54 datafile 2023-12-01T06:39:52.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:52.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj55 datafile 2023-12-01T06:39:52.358 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:52.358 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj56 datafile 2023-12-01T06:39:52.521 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:52.521 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj57 datafile 2023-12-01T06:39:52.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:52.665 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj58 datafile 2023-12-01T06:39:52.808 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:52.808 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj59 datafile 2023-12-01T06:39:52.975 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:52.976 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj60 datafile 2023-12-01T06:39:53.126 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:53.126 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj61 datafile 2023-12-01T06:39:53.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:53.285 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj62 datafile 2023-12-01T06:39:53.427 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:53.427 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj63 datafile 2023-12-01T06:39:53.578 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:53.578 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj64 datafile 2023-12-01T06:39:53.763 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:53.763 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj65 datafile 2023-12-01T06:39:53.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:53.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj66 datafile 2023-12-01T06:39:54.098 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:54.099 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj67 datafile 2023-12-01T06:39:54.258 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:54.258 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj68 datafile 2023-12-01T06:39:54.426 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:54.426 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj69 datafile 2023-12-01T06:39:54.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:54.594 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj70 datafile 2023-12-01T06:39:54.778 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:54.778 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj71 datafile 2023-12-01T06:39:54.929 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:54.929 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj72 datafile 2023-12-01T06:39:55.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:55.089 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj73 datafile 2023-12-01T06:39:55.230 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:55.231 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj74 datafile 2023-12-01T06:39:55.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:55.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj75 datafile 2023-12-01T06:39:55.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:55.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj76 datafile 2023-12-01T06:39:55.700 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:55.700 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj77 datafile 2023-12-01T06:39:55.885 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:55.885 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj78 datafile 2023-12-01T06:39:56.037 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:56.038 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj79 datafile 2023-12-01T06:39:56.211 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:56.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj80 datafile 2023-12-01T06:39:56.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:56.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj81 datafile 2023-12-01T06:39:56.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:56.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj82 datafile 2023-12-01T06:39:56.698 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:56.698 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj83 datafile 2023-12-01T06:39:56.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:56.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj84 datafile 2023-12-01T06:39:56.999 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:56.999 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj85 datafile 2023-12-01T06:39:57.176 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:57.176 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj86 datafile 2023-12-01T06:39:57.355 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:57.355 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj87 datafile 2023-12-01T06:39:57.503 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:57.503 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj88 datafile 2023-12-01T06:39:57.670 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:57.670 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj89 datafile 2023-12-01T06:39:57.846 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:57.846 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj90 datafile 2023-12-01T06:39:57.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:57.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj91 datafile 2023-12-01T06:39:58.149 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:58.149 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj92 datafile 2023-12-01T06:39:58.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:58.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj93 datafile 2023-12-01T06:39:58.458 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:58.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj94 datafile 2023-12-01T06:39:58.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:58.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj95 datafile 2023-12-01T06:39:58.785 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:58.785 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj96 datafile 2023-12-01T06:39:58.936 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:58.936 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj97 datafile 2023-12-01T06:39:59.095 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:59.095 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj98 datafile 2023-12-01T06:39:59.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:59.288 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj99 datafile 2023-12-01T06:39:59.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:59.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj100 datafile 2023-12-01T06:39:59.624 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:59.624 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj101 datafile 2023-12-01T06:39:59.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:59.784 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj102 datafile 2023-12-01T06:39:59.985 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:39:59.985 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj103 datafile 2023-12-01T06:40:00.144 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:00.144 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj104 datafile 2023-12-01T06:40:00.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:00.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj105 datafile 2023-12-01T06:40:00.470 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:00.470 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj106 datafile 2023-12-01T06:40:00.638 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:00.638 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj107 datafile 2023-12-01T06:40:00.797 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:00.797 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj108 datafile 2023-12-01T06:40:00.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:00.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj109 datafile 2023-12-01T06:40:01.100 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:01.100 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj110 datafile 2023-12-01T06:40:01.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:01.276 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj111 datafile 2023-12-01T06:40:01.461 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:01.461 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj112 datafile 2023-12-01T06:40:01.627 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:01.627 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj113 datafile 2023-12-01T06:40:01.803 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:01.804 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj114 datafile 2023-12-01T06:40:01.981 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:01.981 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj115 datafile 2023-12-01T06:40:02.172 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:02.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj116 datafile 2023-12-01T06:40:02.323 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:02.323 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj117 datafile 2023-12-01T06:40:02.482 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:02.482 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj118 datafile 2023-12-01T06:40:02.676 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:02.676 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj119 datafile 2023-12-01T06:40:02.828 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:02.828 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj120 datafile 2023-12-01T06:40:02.977 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:02.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj121 datafile 2023-12-01T06:40:03.104 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:03.104 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj122 datafile 2023-12-01T06:40:03.271 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:03.271 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj123 datafile 2023-12-01T06:40:03.422 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:03.422 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj124 datafile 2023-12-01T06:40:03.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:03.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj125 datafile 2023-12-01T06:40:03.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:03.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj126 datafile 2023-12-01T06:40:03.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:03.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj127 datafile 2023-12-01T06:40:04.144 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:04.144 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj128 datafile 2023-12-01T06:40:04.303 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:04.303 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj129 datafile 2023-12-01T06:40:04.471 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:04.471 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj130 datafile 2023-12-01T06:40:04.621 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:04.621 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj131 datafile 2023-12-01T06:40:04.780 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:04.780 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj132 datafile 2023-12-01T06:40:04.965 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:04.965 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj133 datafile 2023-12-01T06:40:05.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:05.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj134 datafile 2023-12-01T06:40:05.318 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:05.319 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj135 datafile 2023-12-01T06:40:05.484 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:05.485 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj136 datafile 2023-12-01T06:40:05.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:05.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj137 datafile 2023-12-01T06:40:05.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:05.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj138 datafile 2023-12-01T06:40:05.982 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:05.982 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj139 datafile 2023-12-01T06:40:06.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:06.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj140 datafile 2023-12-01T06:40:06.298 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:06.298 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj141 datafile 2023-12-01T06:40:06.465 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:06.465 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj142 datafile 2023-12-01T06:40:06.624 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:06.624 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj143 datafile 2023-12-01T06:40:06.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:06.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj144 datafile 2023-12-01T06:40:06.943 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:06.943 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj145 datafile 2023-12-01T06:40:07.119 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:07.120 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj146 datafile 2023-12-01T06:40:07.279 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:07.279 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj147 datafile 2023-12-01T06:40:07.430 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:07.430 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj148 datafile 2023-12-01T06:40:07.572 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:07.572 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj149 datafile 2023-12-01T06:40:07.873 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:07.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj150 datafile 2023-12-01T06:40:08.058 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:08.058 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj151 datafile 2023-12-01T06:40:08.218 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:08.218 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj152 datafile 2023-12-01T06:40:08.376 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:08.377 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj153 datafile 2023-12-01T06:40:08.536 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:08.536 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj154 datafile 2023-12-01T06:40:08.695 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:08.695 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj155 datafile 2023-12-01T06:40:08.830 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:08.830 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj156 datafile 2023-12-01T06:40:08.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:08.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj157 datafile 2023-12-01T06:40:09.139 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:09.139 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj158 datafile 2023-12-01T06:40:09.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:09.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj159 datafile 2023-12-01T06:40:09.466 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:09.466 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj160 datafile 2023-12-01T06:40:09.642 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:09.642 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj161 datafile 2023-12-01T06:40:09.810 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:09.810 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj162 datafile 2023-12-01T06:40:10.004 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:10.004 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj163 datafile 2023-12-01T06:40:10.154 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:10.154 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj164 datafile 2023-12-01T06:40:10.306 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:10.306 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj165 datafile 2023-12-01T06:40:10.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:10.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj166 datafile 2023-12-01T06:40:10.623 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:10.624 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj167 datafile 2023-12-01T06:40:10.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:10.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj168 datafile 2023-12-01T06:40:10.951 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:10.952 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj169 datafile 2023-12-01T06:40:11.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:11.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj170 datafile 2023-12-01T06:40:11.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:11.278 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj171 datafile 2023-12-01T06:40:11.429 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:11.429 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj172 datafile 2023-12-01T06:40:11.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:11.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj173 datafile 2023-12-01T06:40:11.731 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:11.731 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj174 datafile 2023-12-01T06:40:11.898 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:11.899 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj175 datafile 2023-12-01T06:40:12.067 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:12.067 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj176 datafile 2023-12-01T06:40:12.234 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:12.234 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj177 datafile 2023-12-01T06:40:12.393 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:12.394 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj178 datafile 2023-12-01T06:40:12.552 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:12.552 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj179 datafile 2023-12-01T06:40:12.712 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:12.712 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj180 datafile 2023-12-01T06:40:12.855 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:12.855 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj181 datafile 2023-12-01T06:40:13.014 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:13.014 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj182 datafile 2023-12-01T06:40:13.465 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:13.466 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj183 datafile 2023-12-01T06:40:13.633 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:13.633 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj184 datafile 2023-12-01T06:40:13.828 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:13.828 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj185 datafile 2023-12-01T06:40:13.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:13.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj186 datafile 2023-12-01T06:40:14.163 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:14.163 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj187 datafile 2023-12-01T06:40:14.338 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:14.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj188 datafile 2023-12-01T06:40:14.499 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:14.499 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj189 datafile 2023-12-01T06:40:14.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:14.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj190 datafile 2023-12-01T06:40:14.842 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:14.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj191 datafile 2023-12-01T06:40:15.002 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:15.002 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj192 datafile 2023-12-01T06:40:15.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:15.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj193 datafile 2023-12-01T06:40:15.308 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:15.308 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj194 datafile 2023-12-01T06:40:15.472 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:15.472 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj195 datafile 2023-12-01T06:40:15.630 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:15.631 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj196 datafile 2023-12-01T06:40:15.815 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:15.815 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj197 datafile 2023-12-01T06:40:16.000 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:16.000 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj198 datafile 2023-12-01T06:40:16.201 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:16.202 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj199 datafile 2023-12-01T06:40:16.360 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:16.360 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj200 datafile 2023-12-01T06:40:16.543 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:16.544 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj201 datafile 2023-12-01T06:40:16.695 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:16.695 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj202 datafile 2023-12-01T06:40:16.854 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:16.855 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj203 datafile 2023-12-01T06:40:17.014 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:17.014 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj204 datafile 2023-12-01T06:40:17.164 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:17.165 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj205 datafile 2023-12-01T06:40:17.341 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:17.341 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj206 datafile 2023-12-01T06:40:17.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:17.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj207 datafile 2023-12-01T06:40:17.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:17.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj208 datafile 2023-12-01T06:40:17.860 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:17.860 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj209 datafile 2023-12-01T06:40:18.012 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:18.012 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj210 datafile 2023-12-01T06:40:18.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:18.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj211 datafile 2023-12-01T06:40:18.338 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:18.338 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj212 datafile 2023-12-01T06:40:18.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:18.626 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj213 datafile 2023-12-01T06:40:18.808 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:18.808 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj214 datafile 2023-12-01T06:40:18.991 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:18.991 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj215 datafile 2023-12-01T06:40:19.159 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:19.160 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj216 datafile 2023-12-01T06:40:19.325 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:19.357 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj217 datafile 2023-12-01T06:40:19.486 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:19.486 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj218 datafile 2023-12-01T06:40:19.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:19.645 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj219 datafile 2023-12-01T06:40:19.825 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:19.825 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj220 datafile 2023-12-01T06:40:20.007 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:20.007 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj221 datafile 2023-12-01T06:40:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj222 datafile 2023-12-01T06:40:20.341 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:20.341 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj223 datafile 2023-12-01T06:40:20.518 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:20.518 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj224 datafile 2023-12-01T06:40:20.660 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:20.660 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj225 datafile 2023-12-01T06:40:20.819 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:20.819 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj226 datafile 2023-12-01T06:40:20.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:20.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj227 datafile 2023-12-01T06:40:21.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:21.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj228 datafile 2023-12-01T06:40:21.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:21.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj229 datafile 2023-12-01T06:40:21.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:21.464 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj230 datafile 2023-12-01T06:40:21.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:21.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj231 datafile 2023-12-01T06:40:21.832 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:21.833 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj232 datafile 2023-12-01T06:40:22.009 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:22.009 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj233 datafile 2023-12-01T06:40:22.194 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:22.194 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj234 datafile 2023-12-01T06:40:22.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:22.363 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj235 datafile 2023-12-01T06:40:22.520 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:22.520 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj236 datafile 2023-12-01T06:40:22.672 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:22.672 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj237 datafile 2023-12-01T06:40:22.823 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:22.823 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj238 datafile 2023-12-01T06:40:22.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:22.999 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj239 datafile 2023-12-01T06:40:23.158 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:23.158 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj240 datafile 2023-12-01T06:40:23.332 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:23.333 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj241 datafile 2023-12-01T06:40:23.494 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:23.494 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj242 datafile 2023-12-01T06:40:23.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:23.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj243 datafile 2023-12-01T06:40:23.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:23.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj244 datafile 2023-12-01T06:40:23.996 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:23.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj245 datafile 2023-12-01T06:40:24.156 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:24.156 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj246 datafile 2023-12-01T06:40:24.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:24.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj247 datafile 2023-12-01T06:40:24.482 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:24.483 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj248 datafile 2023-12-01T06:40:24.675 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:24.675 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj249 datafile 2023-12-01T06:40:24.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:24.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj250 datafile 2023-12-01T06:40:25.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:25.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj251 datafile 2023-12-01T06:40:25.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:25.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj252 datafile 2023-12-01T06:40:25.371 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:25.371 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj253 datafile 2023-12-01T06:40:25.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:25.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj254 datafile 2023-12-01T06:40:25.697 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:25.698 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj255 datafile 2023-12-01T06:40:25.883 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:25.883 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj256 datafile 2023-12-01T06:40:26.050 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:26.050 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj257 datafile 2023-12-01T06:40:26.243 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:26.243 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj258 datafile 2023-12-01T06:40:26.402 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:26.402 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj259 datafile 2023-12-01T06:40:26.561 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:26.561 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj260 datafile 2023-12-01T06:40:26.711 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:26.712 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj261 datafile 2023-12-01T06:40:26.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:26.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj262 datafile 2023-12-01T06:40:27.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:27.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj263 datafile 2023-12-01T06:40:27.198 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:27.198 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj264 datafile 2023-12-01T06:40:27.366 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:27.366 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj265 datafile 2023-12-01T06:40:27.541 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:27.541 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj266 datafile 2023-12-01T06:40:27.709 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:27.709 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj267 datafile 2023-12-01T06:40:27.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:27.878 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj268 datafile 2023-12-01T06:40:28.046 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:28.046 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj269 datafile 2023-12-01T06:40:28.213 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:28.213 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj270 datafile 2023-12-01T06:40:28.363 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:28.364 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj271 datafile 2023-12-01T06:40:28.531 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:28.531 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj272 datafile 2023-12-01T06:40:28.691 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:28.692 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj273 datafile 2023-12-01T06:40:28.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:28.858 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj274 datafile 2023-12-01T06:40:29.009 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:29.009 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj275 datafile 2023-12-01T06:40:29.160 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:29.160 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj276 datafile 2023-12-01T06:40:29.352 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:29.352 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj277 datafile 2023-12-01T06:40:29.521 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:29.521 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj278 datafile 2023-12-01T06:40:29.688 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:29.688 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj279 datafile 2023-12-01T06:40:29.855 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:29.856 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj280 datafile 2023-12-01T06:40:30.040 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:30.040 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj281 datafile 2023-12-01T06:40:30.236 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:30.236 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj282 datafile 2023-12-01T06:40:30.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:30.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj283 datafile 2023-12-01T06:40:30.568 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:30.568 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj284 datafile 2023-12-01T06:40:30.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:30.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj285 datafile 2023-12-01T06:40:30.911 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:30.912 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj286 datafile 2023-12-01T06:40:31.063 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:31.063 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj287 datafile 2023-12-01T06:40:31.231 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:31.231 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj288 datafile 2023-12-01T06:40:31.390 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:31.390 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj289 datafile 2023-12-01T06:40:31.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:31.557 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj290 datafile 2023-12-01T06:40:31.700 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:31.700 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj291 datafile 2023-12-01T06:40:31.876 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:31.876 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj292 datafile 2023-12-01T06:40:32.035 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:32.035 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj293 datafile 2023-12-01T06:40:32.187 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:32.187 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj294 datafile 2023-12-01T06:40:32.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:32.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj295 datafile 2023-12-01T06:40:32.496 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:32.497 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj296 datafile 2023-12-01T06:40:32.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:32.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj297 datafile 2023-12-01T06:40:32.841 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:32.841 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj298 datafile 2023-12-01T06:40:33.008 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:33.008 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj299 datafile 2023-12-01T06:40:33.167 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:33.167 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj300 datafile 2023-12-01T06:40:33.326 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:33.326 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj301 datafile 2023-12-01T06:40:33.485 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:33.485 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj302 datafile 2023-12-01T06:40:33.645 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:33.645 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj303 datafile 2023-12-01T06:40:33.854 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:33.855 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj304 datafile 2023-12-01T06:40:34.023 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:34.023 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj305 datafile 2023-12-01T06:40:34.190 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:34.191 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj306 datafile 2023-12-01T06:40:34.358 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:34.358 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj307 datafile 2023-12-01T06:40:34.538 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:34.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj308 datafile 2023-12-01T06:40:34.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:34.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj309 datafile 2023-12-01T06:40:34.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:34.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj310 datafile 2023-12-01T06:40:34.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:34.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj311 datafile 2023-12-01T06:40:35.155 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:35.155 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj312 datafile 2023-12-01T06:40:35.322 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:35.322 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj313 datafile 2023-12-01T06:40:35.498 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:35.498 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj314 datafile 2023-12-01T06:40:35.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:35.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj315 datafile 2023-12-01T06:40:35.875 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:35.875 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj316 datafile 2023-12-01T06:40:36.043 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:36.043 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj317 datafile 2023-12-01T06:40:36.219 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:36.219 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj318 datafile 2023-12-01T06:40:36.370 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:36.370 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj319 datafile 2023-12-01T06:40:36.521 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:36.521 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj320 datafile 2023-12-01T06:40:36.689 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:36.689 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj321 datafile 2023-12-01T06:40:36.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:36.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj322 datafile 2023-12-01T06:40:37.040 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:37.040 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj323 datafile 2023-12-01T06:40:37.233 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:37.233 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj324 datafile 2023-12-01T06:40:37.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:37.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj325 datafile 2023-12-01T06:40:37.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:37.595 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj326 datafile 2023-12-01T06:40:37.736 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:37.736 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj327 datafile 2023-12-01T06:40:37.914 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:37.915 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj328 datafile 2023-12-01T06:40:38.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:38.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj329 datafile 2023-12-01T06:40:38.223 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:38.223 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj330 datafile 2023-12-01T06:40:38.390 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:38.390 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj331 datafile 2023-12-01T06:40:38.541 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:38.542 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj332 datafile 2023-12-01T06:40:38.726 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:38.726 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj333 datafile 2023-12-01T06:40:38.894 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:38.895 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj334 datafile 2023-12-01T06:40:39.111 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:39.112 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj335 datafile 2023-12-01T06:40:39.254 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:39.254 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj336 datafile 2023-12-01T06:40:39.422 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:39.422 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj337 datafile 2023-12-01T06:40:39.572 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:39.572 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj338 datafile 2023-12-01T06:40:39.763 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:39.763 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj339 datafile 2023-12-01T06:40:40.003 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:40.004 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj340 datafile 2023-12-01T06:40:40.202 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:40.203 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj341 datafile 2023-12-01T06:40:40.377 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:40.377 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj342 datafile 2023-12-01T06:40:40.520 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:40.520 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj343 datafile 2023-12-01T06:40:40.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:40.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj344 datafile 2023-12-01T06:40:40.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:40.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj345 datafile 2023-12-01T06:40:40.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:40.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj346 datafile 2023-12-01T06:40:41.157 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:41.157 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj347 datafile 2023-12-01T06:40:41.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:41.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj348 datafile 2023-12-01T06:40:41.534 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:41.534 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj349 datafile 2023-12-01T06:40:41.710 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:41.710 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj350 datafile 2023-12-01T06:40:41.904 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:41.905 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj351 datafile 2023-12-01T06:40:42.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:42.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj352 datafile 2023-12-01T06:40:42.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:42.239 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj353 datafile 2023-12-01T06:40:42.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:42.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj354 datafile 2023-12-01T06:40:42.565 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:42.565 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj355 datafile 2023-12-01T06:40:42.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:42.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj356 datafile 2023-12-01T06:40:42.909 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:42.909 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj357 datafile 2023-12-01T06:40:43.068 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:43.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj358 datafile 2023-12-01T06:40:43.272 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:43.272 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj359 datafile 2023-12-01T06:40:43.437 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:43.437 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj360 datafile 2023-12-01T06:40:43.614 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:43.614 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj361 datafile 2023-12-01T06:40:43.798 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:43.798 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj362 datafile 2023-12-01T06:40:43.992 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:43.992 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj363 datafile 2023-12-01T06:40:44.425 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:44.426 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj364 datafile 2023-12-01T06:40:44.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:44.592 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj365 datafile 2023-12-01T06:40:44.752 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:44.752 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj366 datafile 2023-12-01T06:40:44.911 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:44.947 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj367 datafile 2023-12-01T06:40:45.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:45.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj368 datafile 2023-12-01T06:40:45.247 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:45.247 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj369 datafile 2023-12-01T06:40:45.406 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:45.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj370 datafile 2023-12-01T06:40:45.565 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:45.566 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj371 datafile 2023-12-01T06:40:45.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:45.717 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj372 datafile 2023-12-01T06:40:45.910 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:45.910 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj373 datafile 2023-12-01T06:40:46.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:46.070 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj374 datafile 2023-12-01T06:40:46.245 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:46.245 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj375 datafile 2023-12-01T06:40:46.396 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:46.396 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj376 datafile 2023-12-01T06:40:46.555 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:46.555 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj377 datafile 2023-12-01T06:40:46.723 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:46.724 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj378 datafile 2023-12-01T06:40:46.882 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:46.882 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj379 datafile 2023-12-01T06:40:47.035 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:47.035 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj380 datafile 2023-12-01T06:40:47.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:47.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj381 datafile 2023-12-01T06:40:47.377 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:47.377 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj382 datafile 2023-12-01T06:40:47.528 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:47.528 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj383 datafile 2023-12-01T06:40:47.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:47.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj384 datafile 2023-12-01T06:40:47.855 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:47.855 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj385 datafile 2023-12-01T06:40:48.023 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:48.023 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj386 datafile 2023-12-01T06:40:48.190 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:48.190 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj387 datafile 2023-12-01T06:40:48.358 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:48.358 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj388 datafile 2023-12-01T06:40:48.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:48.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj389 datafile 2023-12-01T06:40:48.710 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:48.710 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj390 datafile 2023-12-01T06:40:48.894 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:48.894 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj391 datafile 2023-12-01T06:40:49.054 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:49.054 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj392 datafile 2023-12-01T06:40:49.221 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:49.221 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj393 datafile 2023-12-01T06:40:49.398 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:49.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj394 datafile 2023-12-01T06:40:49.640 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:49.640 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj395 datafile 2023-12-01T06:40:49.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:49.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj396 datafile 2023-12-01T06:40:50.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:50.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj397 datafile 2023-12-01T06:40:50.194 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:50.194 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj398 datafile 2023-12-01T06:40:50.370 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:50.370 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj399 datafile 2023-12-01T06:40:50.538 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:50.538 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj400 datafile 2023-12-01T06:40:50.696 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:50.697 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj401 datafile 2023-12-01T06:40:50.856 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:50.856 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj402 datafile 2023-12-01T06:40:51.032 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:51.032 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj403 datafile 2023-12-01T06:40:51.199 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:51.200 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj404 datafile 2023-12-01T06:40:51.402 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:51.402 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj405 datafile 2023-12-01T06:40:51.575 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:51.576 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj406 datafile 2023-12-01T06:40:51.735 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:51.735 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj407 datafile 2023-12-01T06:40:51.929 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:51.929 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj408 datafile 2023-12-01T06:40:52.088 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:52.088 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj409 datafile 2023-12-01T06:40:52.248 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:52.248 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj410 datafile 2023-12-01T06:40:52.423 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:52.423 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj411 datafile 2023-12-01T06:40:52.599 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:52.599 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj412 datafile 2023-12-01T06:40:52.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:52.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj413 datafile 2023-12-01T06:40:52.960 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:52.960 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj414 datafile 2023-12-01T06:40:53.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:53.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj415 datafile 2023-12-01T06:40:53.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:53.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj416 datafile 2023-12-01T06:40:53.455 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:53.455 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj417 datafile 2023-12-01T06:40:53.621 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:53.622 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj418 datafile 2023-12-01T06:40:53.790 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:53.790 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj419 datafile 2023-12-01T06:40:53.974 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:53.975 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj420 datafile 2023-12-01T06:40:54.142 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:54.142 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj421 datafile 2023-12-01T06:40:54.334 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:54.334 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj422 datafile 2023-12-01T06:40:54.518 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:54.518 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj423 datafile 2023-12-01T06:40:54.686 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:54.686 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj424 datafile 2023-12-01T06:40:54.862 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:54.862 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj425 datafile 2023-12-01T06:40:55.021 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:55.021 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj426 datafile 2023-12-01T06:40:55.190 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:55.190 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj427 datafile 2023-12-01T06:40:55.374 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:55.374 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj428 datafile 2023-12-01T06:40:55.550 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:55.550 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj429 datafile 2023-12-01T06:40:55.709 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:55.709 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj430 datafile 2023-12-01T06:40:55.885 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:55.886 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj431 datafile 2023-12-01T06:40:56.045 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:56.045 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj432 datafile 2023-12-01T06:40:56.195 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:56.196 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj433 datafile 2023-12-01T06:40:56.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:56.349 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj434 datafile 2023-12-01T06:40:56.497 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:56.497 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj435 datafile 2023-12-01T06:40:56.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:56.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj436 datafile 2023-12-01T06:40:56.832 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:56.832 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj437 datafile 2023-12-01T06:40:56.975 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:56.975 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj438 datafile 2023-12-01T06:40:57.118 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:57.118 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj439 datafile 2023-12-01T06:40:57.293 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:57.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj440 datafile 2023-12-01T06:40:57.453 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:57.453 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj441 datafile 2023-12-01T06:40:57.622 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:57.623 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj442 datafile 2023-12-01T06:40:57.816 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:57.816 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj443 datafile 2023-12-01T06:40:58.017 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:58.017 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj444 datafile 2023-12-01T06:40:58.168 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:58.168 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj445 datafile 2023-12-01T06:40:58.327 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:58.327 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj446 datafile 2023-12-01T06:40:58.494 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:58.494 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj447 datafile 2023-12-01T06:40:58.653 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:58.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj448 datafile 2023-12-01T06:40:58.804 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:58.805 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj449 datafile 2023-12-01T06:40:58.964 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:58.964 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj450 datafile 2023-12-01T06:40:59.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:59.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj451 datafile 2023-12-01T06:40:59.299 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:59.299 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj452 datafile 2023-12-01T06:40:59.467 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:59.467 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj453 datafile 2023-12-01T06:40:59.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:59.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj454 datafile 2023-12-01T06:40:59.785 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:59.785 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj455 datafile 2023-12-01T06:40:59.979 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:40:59.979 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj456 datafile 2023-12-01T06:41:00.142 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:00.143 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj457 datafile 2023-12-01T06:41:00.322 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:00.322 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj458 datafile 2023-12-01T06:41:00.490 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:00.490 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj459 datafile 2023-12-01T06:41:00.640 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:00.641 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj460 datafile 2023-12-01T06:41:00.808 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:00.808 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj461 datafile 2023-12-01T06:41:00.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:00.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj462 datafile 2023-12-01T06:41:01.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:01.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj463 datafile 2023-12-01T06:41:01.336 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:01.337 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj464 datafile 2023-12-01T06:41:01.490 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:01.490 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj465 datafile 2023-12-01T06:41:01.648 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:01.648 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj466 datafile 2023-12-01T06:41:01.818 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:01.819 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj467 datafile 2023-12-01T06:41:01.999 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:01.999 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj468 datafile 2023-12-01T06:41:02.166 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:02.167 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj469 datafile 2023-12-01T06:41:02.318 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:02.318 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj470 datafile 2023-12-01T06:41:02.468 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:02.469 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj471 datafile 2023-12-01T06:41:02.628 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:02.628 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj472 datafile 2023-12-01T06:41:02.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:02.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj473 datafile 2023-12-01T06:41:02.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:02.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj474 datafile 2023-12-01T06:41:03.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:03.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj475 datafile 2023-12-01T06:41:03.336 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:03.336 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj476 datafile 2023-12-01T06:41:03.483 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:03.483 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj477 datafile 2023-12-01T06:41:03.651 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:03.651 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj478 datafile 2023-12-01T06:41:03.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:03.844 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj479 datafile 2023-12-01T06:41:04.027 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:04.027 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj480 datafile 2023-12-01T06:41:04.178 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:04.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj481 datafile 2023-12-01T06:41:04.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:04.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj482 datafile 2023-12-01T06:41:04.498 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:04.499 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj483 datafile 2023-12-01T06:41:04.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:04.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj484 datafile 2023-12-01T06:41:04.873 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:04.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj485 datafile 2023-12-01T06:41:05.025 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:05.025 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj486 datafile 2023-12-01T06:41:05.175 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:05.176 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj487 datafile 2023-12-01T06:41:05.335 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:05.335 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj488 datafile 2023-12-01T06:41:05.486 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:05.486 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj489 datafile 2023-12-01T06:41:05.628 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:05.628 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj490 datafile 2023-12-01T06:41:05.796 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:05.796 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj491 datafile 2023-12-01T06:41:05.981 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:05.982 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj492 datafile 2023-12-01T06:41:06.149 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:06.149 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj493 datafile 2023-12-01T06:41:06.299 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:06.299 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj494 datafile 2023-12-01T06:41:06.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:06.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj495 datafile 2023-12-01T06:41:06.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:06.635 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj496 datafile 2023-12-01T06:41:06.810 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:06.810 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj497 datafile 2023-12-01T06:41:06.979 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:06.979 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj498 datafile 2023-12-01T06:41:07.137 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:07.138 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj499 datafile 2023-12-01T06:41:07.322 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:07.323 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj500 datafile 2023-12-01T06:41:07.499 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:07.499 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj501 datafile 2023-12-01T06:41:07.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:07.666 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj502 datafile 2023-12-01T06:41:07.834 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:07.834 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj503 datafile 2023-12-01T06:41:08.018 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:08.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj504 datafile 2023-12-01T06:41:08.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:08.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj505 datafile 2023-12-01T06:41:08.337 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:08.337 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj506 datafile 2023-12-01T06:41:08.488 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:08.488 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj507 datafile 2023-12-01T06:41:08.655 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:08.656 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj508 datafile 2023-12-01T06:41:08.806 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:08.806 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj509 datafile 2023-12-01T06:41:08.957 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:08.957 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj510 datafile 2023-12-01T06:41:09.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:09.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj511 datafile 2023-12-01T06:41:09.292 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:09.292 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj512 datafile 2023-12-01T06:41:09.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:09.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj513 datafile 2023-12-01T06:41:09.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:09.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj514 datafile 2023-12-01T06:41:09.803 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:09.803 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj515 datafile 2023-12-01T06:41:10.065 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:10.066 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj516 datafile 2023-12-01T06:41:10.248 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:10.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj517 datafile 2023-12-01T06:41:10.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:10.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj518 datafile 2023-12-01T06:41:10.584 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:10.584 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj519 datafile 2023-12-01T06:41:10.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:10.744 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj520 datafile 2023-12-01T06:41:10.902 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:10.902 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj521 datafile 2023-12-01T06:41:11.062 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:11.062 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj522 datafile 2023-12-01T06:41:11.221 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:11.221 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj523 datafile 2023-12-01T06:41:11.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:11.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj524 datafile 2023-12-01T06:41:11.555 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:11.555 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj525 datafile 2023-12-01T06:41:11.706 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:11.707 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj526 datafile 2023-12-01T06:41:11.916 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:11.916 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj527 datafile 2023-12-01T06:41:12.067 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:12.067 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj528 datafile 2023-12-01T06:41:12.219 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:12.219 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj529 datafile 2023-12-01T06:41:12.378 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:12.378 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj530 datafile 2023-12-01T06:41:12.545 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:12.545 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj531 datafile 2023-12-01T06:41:12.704 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:12.705 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj532 datafile 2023-12-01T06:41:12.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:12.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj533 datafile 2023-12-01T06:41:13.050 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:13.050 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj534 datafile 2023-12-01T06:41:13.216 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:13.216 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj535 datafile 2023-12-01T06:41:13.392 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:13.392 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj536 datafile 2023-12-01T06:41:13.543 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:13.543 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj537 datafile 2023-12-01T06:41:13.711 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:13.711 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj538 datafile 2023-12-01T06:41:13.895 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:13.896 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj539 datafile 2023-12-01T06:41:14.063 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:14.063 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj540 datafile 2023-12-01T06:41:14.247 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:14.247 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj541 datafile 2023-12-01T06:41:14.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:14.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj542 datafile 2023-12-01T06:41:14.582 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:14.583 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj543 datafile 2023-12-01T06:41:14.741 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:14.741 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj544 datafile 2023-12-01T06:41:14.893 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:14.893 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj545 datafile 2023-12-01T06:41:15.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:15.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj546 datafile 2023-12-01T06:41:15.455 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:15.455 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj547 datafile 2023-12-01T06:41:15.621 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:15.658 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj548 datafile 2023-12-01T06:41:15.814 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:15.814 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj549 datafile 2023-12-01T06:41:16.024 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:16.025 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj550 datafile 2023-12-01T06:41:16.191 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:16.191 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj551 datafile 2023-12-01T06:41:16.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:16.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj552 datafile 2023-12-01T06:41:16.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:16.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj553 datafile 2023-12-01T06:41:16.660 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:16.660 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj554 datafile 2023-12-01T06:41:16.819 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:16.819 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj555 datafile 2023-12-01T06:41:16.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:16.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj556 datafile 2023-12-01T06:41:17.155 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:17.155 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj557 datafile 2023-12-01T06:41:17.323 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:17.323 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj558 datafile 2023-12-01T06:41:17.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:17.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj559 datafile 2023-12-01T06:41:17.624 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:17.624 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj560 datafile 2023-12-01T06:41:17.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:17.792 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj561 datafile 2023-12-01T06:41:17.976 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:17.977 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj562 datafile 2023-12-01T06:41:18.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:18.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj563 datafile 2023-12-01T06:41:18.328 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:18.328 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj564 datafile 2023-12-01T06:41:18.480 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:18.480 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj565 datafile 2023-12-01T06:41:18.613 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:18.614 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj566 datafile 2023-12-01T06:41:18.775 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:18.776 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj567 datafile 2023-12-01T06:41:18.974 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:18.975 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj568 datafile 2023-12-01T06:41:19.133 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:19.133 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj569 datafile 2023-12-01T06:41:19.293 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:19.293 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj570 datafile 2023-12-01T06:41:19.452 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:19.452 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj571 datafile 2023-12-01T06:41:19.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:19.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj572 datafile 2023-12-01T06:41:19.754 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:19.754 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj573 datafile 2023-12-01T06:41:19.905 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:19.905 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj574 datafile 2023-12-01T06:41:20.065 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:20.065 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj575 datafile 2023-12-01T06:41:20.224 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:20.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj576 datafile 2023-12-01T06:41:20.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:20.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj577 datafile 2023-12-01T06:41:20.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:20.619 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj578 datafile 2023-12-01T06:41:20.777 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:20.777 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj579 datafile 2023-12-01T06:41:20.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:20.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj580 datafile 2023-12-01T06:41:21.136 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:21.136 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj581 datafile 2023-12-01T06:41:21.296 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:21.297 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj582 datafile 2023-12-01T06:41:21.532 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:21.532 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj583 datafile 2023-12-01T06:41:21.741 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:21.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj584 datafile 2023-12-01T06:41:21.977 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:21.977 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj585 datafile 2023-12-01T06:41:22.145 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:22.145 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj586 datafile 2023-12-01T06:41:22.303 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:22.304 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj587 datafile 2023-12-01T06:41:22.472 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:22.472 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj588 datafile 2023-12-01T06:41:22.639 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:22.639 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj589 datafile 2023-12-01T06:41:22.798 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:22.798 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj590 datafile 2023-12-01T06:41:22.957 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:22.957 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj591 datafile 2023-12-01T06:41:23.108 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:23.108 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj592 datafile 2023-12-01T06:41:23.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:23.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj593 datafile 2023-12-01T06:41:23.443 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:23.444 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj594 datafile 2023-12-01T06:41:23.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:23.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj595 datafile 2023-12-01T06:41:23.804 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:23.804 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj596 datafile 2023-12-01T06:41:23.989 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:23.989 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj597 datafile 2023-12-01T06:41:24.156 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:24.157 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj598 datafile 2023-12-01T06:41:24.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:24.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj599 datafile 2023-12-01T06:41:24.467 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:24.467 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj600 datafile 2023-12-01T06:41:24.634 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:24.634 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj601 datafile 2023-12-01T06:41:24.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:24.793 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj602 datafile 2023-12-01T06:41:24.970 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:24.970 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj603 datafile 2023-12-01T06:41:25.153 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:25.153 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj604 datafile 2023-12-01T06:41:25.322 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:25.322 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj605 datafile 2023-12-01T06:41:25.490 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:25.490 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj606 datafile 2023-12-01T06:41:25.667 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:25.667 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj607 datafile 2023-12-01T06:41:25.850 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:25.850 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj608 datafile 2023-12-01T06:41:26.001 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:26.001 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj609 datafile 2023-12-01T06:41:26.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:26.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj610 datafile 2023-12-01T06:41:26.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:26.294 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj611 datafile 2023-12-01T06:41:26.436 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:26.436 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj612 datafile 2023-12-01T06:41:26.613 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:26.613 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj613 datafile 2023-12-01T06:41:26.757 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:26.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj614 datafile 2023-12-01T06:41:26.914 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:26.915 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj615 datafile 2023-12-01T06:41:27.074 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:27.074 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj616 datafile 2023-12-01T06:41:27.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:27.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj617 datafile 2023-12-01T06:41:27.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:27.384 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj618 datafile 2023-12-01T06:41:27.544 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:27.544 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj619 datafile 2023-12-01T06:41:27.694 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:27.694 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj620 datafile 2023-12-01T06:41:27.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:27.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj621 datafile 2023-12-01T06:41:28.047 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:28.048 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj622 datafile 2023-12-01T06:41:28.207 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:28.207 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj623 datafile 2023-12-01T06:41:28.365 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:28.366 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj624 datafile 2023-12-01T06:41:28.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:28.525 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj625 datafile 2023-12-01T06:41:28.692 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:28.692 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj626 datafile 2023-12-01T06:41:28.844 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:28.844 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj627 datafile 2023-12-01T06:41:29.003 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:29.003 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj628 datafile 2023-12-01T06:41:29.162 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:29.162 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj629 datafile 2023-12-01T06:41:29.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:29.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj630 datafile 2023-12-01T06:41:29.481 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:29.481 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj631 datafile 2023-12-01T06:41:29.648 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:29.648 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj632 datafile 2023-12-01T06:41:29.833 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:29.834 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj633 datafile 2023-12-01T06:41:30.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:30.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj634 datafile 2023-12-01T06:41:30.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:30.177 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj635 datafile 2023-12-01T06:41:30.377 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:30.377 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj636 datafile 2023-12-01T06:41:30.537 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:30.537 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj637 datafile 2023-12-01T06:41:30.705 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:30.706 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj638 datafile 2023-12-01T06:41:30.863 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:30.863 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj639 datafile 2023-12-01T06:41:31.015 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:31.015 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj640 datafile 2023-12-01T06:41:31.166 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:31.166 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj641 datafile 2023-12-01T06:41:31.350 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:31.350 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj642 datafile 2023-12-01T06:41:31.502 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:31.502 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj643 datafile 2023-12-01T06:41:31.669 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:31.669 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj644 datafile 2023-12-01T06:41:31.844 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:31.844 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj645 datafile 2023-12-01T06:41:32.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:32.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj646 datafile 2023-12-01T06:41:32.205 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:32.205 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj647 datafile 2023-12-01T06:41:32.373 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:32.373 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj648 datafile 2023-12-01T06:41:32.523 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:32.523 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj649 datafile 2023-12-01T06:41:32.682 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:32.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj650 datafile 2023-12-01T06:41:32.833 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:32.834 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj651 datafile 2023-12-01T06:41:32.984 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:32.984 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj652 datafile 2023-12-01T06:41:33.135 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:33.135 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj653 datafile 2023-12-01T06:41:33.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:33.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj654 datafile 2023-12-01T06:41:33.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:33.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj655 datafile 2023-12-01T06:41:33.621 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:33.621 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj656 datafile 2023-12-01T06:41:33.770 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:33.770 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj657 datafile 2023-12-01T06:41:33.950 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:33.951 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj658 datafile 2023-12-01T06:41:34.101 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:34.102 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj659 datafile 2023-12-01T06:41:34.242 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:34.243 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj660 datafile 2023-12-01T06:41:34.402 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:34.402 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj661 datafile 2023-12-01T06:41:34.561 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:34.561 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj662 datafile 2023-12-01T06:41:34.746 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:34.746 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj663 datafile 2023-12-01T06:41:34.896 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:34.896 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj664 datafile 2023-12-01T06:41:35.080 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:35.081 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj665 datafile 2023-12-01T06:41:35.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:35.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj666 datafile 2023-12-01T06:41:35.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:35.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj667 datafile 2023-12-01T06:41:35.550 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:35.551 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj668 datafile 2023-12-01T06:41:35.772 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:35.772 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj669 datafile 2023-12-01T06:41:35.980 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:35.980 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj670 datafile 2023-12-01T06:41:36.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:36.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj671 datafile 2023-12-01T06:41:36.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:36.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj672 datafile 2023-12-01T06:41:36.498 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:36.498 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj673 datafile 2023-12-01T06:41:36.649 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:36.649 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj674 datafile 2023-12-01T06:41:36.817 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:36.817 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj675 datafile 2023-12-01T06:41:36.994 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:36.994 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj676 datafile 2023-12-01T06:41:37.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:37.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj677 datafile 2023-12-01T06:41:37.328 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:37.329 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj678 datafile 2023-12-01T06:41:37.496 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:37.496 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj679 datafile 2023-12-01T06:41:37.663 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:37.664 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj680 datafile 2023-12-01T06:41:37.865 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:37.865 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj681 datafile 2023-12-01T06:41:38.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:38.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj682 datafile 2023-12-01T06:41:38.192 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:38.193 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj683 datafile 2023-12-01T06:41:38.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:38.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj684 datafile 2023-12-01T06:41:38.502 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:38.502 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj685 datafile 2023-12-01T06:41:38.670 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:38.670 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj686 datafile 2023-12-01T06:41:38.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:38.821 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj687 datafile 2023-12-01T06:41:38.971 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:38.971 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj688 datafile 2023-12-01T06:41:39.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:39.131 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj689 datafile 2023-12-01T06:41:39.306 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:39.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj690 datafile 2023-12-01T06:41:39.458 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:39.458 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj691 datafile 2023-12-01T06:41:39.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:39.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj692 datafile 2023-12-01T06:41:39.768 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:39.768 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj693 datafile 2023-12-01T06:41:40.011 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:40.012 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj694 datafile 2023-12-01T06:41:40.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:40.171 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj695 datafile 2023-12-01T06:41:40.329 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:40.330 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj696 datafile 2023-12-01T06:41:40.481 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:40.481 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj697 datafile 2023-12-01T06:41:40.676 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:40.676 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj698 datafile 2023-12-01T06:41:40.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:40.843 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj699 datafile 2023-12-01T06:41:41.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:41.069 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj700 datafile 2023-12-01T06:41:41.245 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:41.245 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj701 datafile 2023-12-01T06:41:41.413 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:41.414 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj702 datafile 2023-12-01T06:41:41.572 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:41.572 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj703 datafile 2023-12-01T06:41:41.740 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:41.740 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj704 datafile 2023-12-01T06:41:42.150 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:42.150 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj705 datafile 2023-12-01T06:41:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:42.301 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj706 datafile 2023-12-01T06:41:42.469 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:42.469 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj707 datafile 2023-12-01T06:41:42.628 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:42.628 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj708 datafile 2023-12-01T06:41:42.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:42.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj709 datafile 2023-12-01T06:41:42.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:42.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj710 datafile 2023-12-01T06:41:43.122 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:43.123 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj711 datafile 2023-12-01T06:41:43.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:43.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj712 datafile 2023-12-01T06:41:43.492 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:43.492 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj713 datafile 2023-12-01T06:41:43.642 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:43.642 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj714 datafile 2023-12-01T06:41:43.828 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:43.828 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj715 datafile 2023-12-01T06:41:44.013 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:44.013 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj716 datafile 2023-12-01T06:41:44.162 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:44.163 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj717 datafile 2023-12-01T06:41:44.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:44.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj718 datafile 2023-12-01T06:41:44.482 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:44.482 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj719 datafile 2023-12-01T06:41:44.649 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:44.649 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj720 datafile 2023-12-01T06:41:44.817 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:44.817 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj721 datafile 2023-12-01T06:41:44.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:44.968 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj722 datafile 2023-12-01T06:41:45.135 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:45.135 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj723 datafile 2023-12-01T06:41:45.295 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:45.295 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj724 datafile 2023-12-01T06:41:45.454 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:45.454 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj725 datafile 2023-12-01T06:41:45.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:45.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj726 datafile 2023-12-01T06:41:45.756 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:45.756 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj727 datafile 2023-12-01T06:41:45.957 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:45.957 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj728 datafile 2023-12-01T06:41:46.300 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:46.300 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj729 datafile 2023-12-01T06:41:46.452 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:46.452 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj730 datafile 2023-12-01T06:41:46.585 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:46.586 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj731 datafile 2023-12-01T06:41:46.745 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:46.745 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj732 datafile 2023-12-01T06:41:46.904 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:46.904 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj733 datafile 2023-12-01T06:41:47.097 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:47.098 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj734 datafile 2023-12-01T06:41:47.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:47.291 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj735 datafile 2023-12-01T06:41:47.448 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:47.449 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj736 datafile 2023-12-01T06:41:47.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:47.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj737 datafile 2023-12-01T06:41:47.776 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:47.777 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj738 datafile 2023-12-01T06:41:47.969 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:47.970 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj739 datafile 2023-12-01T06:41:48.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:48.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj740 datafile 2023-12-01T06:41:48.271 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:48.271 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj741 datafile 2023-12-01T06:41:48.430 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:48.430 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj742 datafile 2023-12-01T06:41:48.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:48.581 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj743 datafile 2023-12-01T06:41:48.741 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:48.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj744 datafile 2023-12-01T06:41:48.925 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:48.925 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj745 datafile 2023-12-01T06:41:49.109 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:49.109 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj746 datafile 2023-12-01T06:41:49.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:49.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj747 datafile 2023-12-01T06:41:49.420 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:49.420 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj748 datafile 2023-12-01T06:41:49.579 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:49.579 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj749 datafile 2023-12-01T06:41:49.738 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:49.738 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj750 datafile 2023-12-01T06:41:49.881 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:49.881 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj751 datafile 2023-12-01T06:41:50.040 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:50.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj752 datafile 2023-12-01T06:41:50.208 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:50.209 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj753 datafile 2023-12-01T06:41:50.367 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:50.368 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj754 datafile 2023-12-01T06:41:50.526 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:50.527 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj755 datafile 2023-12-01T06:41:50.678 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:50.678 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj756 datafile 2023-12-01T06:41:50.838 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:50.838 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj757 datafile 2023-12-01T06:41:50.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:50.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj758 datafile 2023-12-01T06:41:51.306 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:51.306 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj759 datafile 2023-12-01T06:41:51.473 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:51.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj760 datafile 2023-12-01T06:41:51.625 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:51.625 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj761 datafile 2023-12-01T06:41:51.801 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:51.801 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj762 datafile 2023-12-01T06:41:52.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:52.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj763 datafile 2023-12-01T06:41:52.229 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:52.230 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj764 datafile 2023-12-01T06:41:52.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:52.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj765 datafile 2023-12-01T06:41:52.563 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:52.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj766 datafile 2023-12-01T06:41:52.734 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:52.734 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj767 datafile 2023-12-01T06:41:52.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:52.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj768 datafile 2023-12-01T06:41:53.042 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:53.042 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj769 datafile 2023-12-01T06:41:53.193 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:53.193 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj770 datafile 2023-12-01T06:41:53.343 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:53.343 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj771 datafile 2023-12-01T06:41:53.495 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:53.495 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj772 datafile 2023-12-01T06:41:53.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:53.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj773 datafile 2023-12-01T06:41:53.832 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:53.832 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj774 datafile 2023-12-01T06:41:54.015 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:54.015 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj775 datafile 2023-12-01T06:41:54.181 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:54.181 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj776 datafile 2023-12-01T06:41:54.333 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:54.333 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj777 datafile 2023-12-01T06:41:54.484 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:54.484 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj778 datafile 2023-12-01T06:41:54.651 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:54.651 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj779 datafile 2023-12-01T06:41:54.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:54.827 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj780 datafile 2023-12-01T06:41:54.995 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:54.995 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj781 datafile 2023-12-01T06:41:55.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:55.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj782 datafile 2023-12-01T06:41:55.322 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:55.322 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj783 datafile 2023-12-01T06:41:55.482 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:55.482 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj784 datafile 2023-12-01T06:41:55.632 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:55.632 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj785 datafile 2023-12-01T06:41:55.817 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:55.817 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj786 datafile 2023-12-01T06:41:56.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:56.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj787 datafile 2023-12-01T06:41:56.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:56.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj788 datafile 2023-12-01T06:41:56.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:56.345 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj789 datafile 2023-12-01T06:41:56.496 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:56.496 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj790 datafile 2023-12-01T06:41:56.655 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:56.655 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj791 datafile 2023-12-01T06:41:56.823 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:56.823 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj792 datafile 2023-12-01T06:41:56.974 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:56.974 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj793 datafile 2023-12-01T06:41:57.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:57.124 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj794 datafile 2023-12-01T06:41:57.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:57.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj795 datafile 2023-12-01T06:41:57.427 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:57.427 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj796 datafile 2023-12-01T06:41:57.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:57.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj797 datafile 2023-12-01T06:41:57.754 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:57.754 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj798 datafile 2023-12-01T06:41:57.931 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:57.931 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj799 datafile 2023-12-01T06:41:58.106 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:58.106 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj800 datafile 2023-12-01T06:41:58.257 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:58.257 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj801 datafile 2023-12-01T06:41:58.400 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:58.400 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj802 datafile 2023-12-01T06:41:58.567 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:58.567 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj803 datafile 2023-12-01T06:41:58.718 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:58.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj804 datafile 2023-12-01T06:41:58.869 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:58.869 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj805 datafile 2023-12-01T06:41:59.021 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:59.021 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj806 datafile 2023-12-01T06:41:59.188 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:59.188 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj807 datafile 2023-12-01T06:41:59.364 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:59.364 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj808 datafile 2023-12-01T06:41:59.531 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:59.532 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj809 datafile 2023-12-01T06:41:59.691 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:59.691 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj810 datafile 2023-12-01T06:41:59.875 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:41:59.875 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj811 datafile 2023-12-01T06:42:00.035 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:00.036 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj812 datafile 2023-12-01T06:42:00.186 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:00.186 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj813 datafile 2023-12-01T06:42:00.370 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:00.370 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj814 datafile 2023-12-01T06:42:00.521 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:00.521 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj815 datafile 2023-12-01T06:42:00.689 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:00.689 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj816 datafile 2023-12-01T06:42:00.848 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:00.848 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj817 datafile 2023-12-01T06:42:01.016 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:01.016 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj818 datafile 2023-12-01T06:42:01.184 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:01.184 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj819 datafile 2023-12-01T06:42:01.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:01.348 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj820 datafile 2023-12-01T06:42:01.511 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:01.511 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj821 datafile 2023-12-01T06:42:01.686 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:01.686 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj822 datafile 2023-12-01T06:42:01.838 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:01.838 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj823 datafile 2023-12-01T06:42:01.989 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:01.989 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj824 datafile 2023-12-01T06:42:02.139 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:02.140 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj825 datafile 2023-12-01T06:42:02.282 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:02.282 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj826 datafile 2023-12-01T06:42:02.443 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:02.443 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj827 datafile 2023-12-01T06:42:02.609 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:02.609 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj828 datafile 2023-12-01T06:42:02.760 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:02.760 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj829 datafile 2023-12-01T06:42:02.911 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:02.911 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj830 datafile 2023-12-01T06:42:03.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:03.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj831 datafile 2023-12-01T06:42:03.254 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:03.254 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj832 datafile 2023-12-01T06:42:03.431 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:03.431 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj833 datafile 2023-12-01T06:42:03.590 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:03.590 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj834 datafile 2023-12-01T06:42:03.749 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:03.749 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj835 datafile 2023-12-01T06:42:03.933 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:03.933 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj836 datafile 2023-12-01T06:42:04.101 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:04.102 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj837 datafile 2023-12-01T06:42:04.253 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:04.253 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj838 datafile 2023-12-01T06:42:04.429 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:04.429 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj839 datafile 2023-12-01T06:42:04.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:04.588 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj840 datafile 2023-12-01T06:42:04.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:04.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj841 datafile 2023-12-01T06:42:04.907 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:04.907 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj842 datafile 2023-12-01T06:42:05.066 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:05.066 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj843 datafile 2023-12-01T06:42:05.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:05.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj844 datafile 2023-12-01T06:42:05.418 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:05.418 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj845 datafile 2023-12-01T06:42:05.577 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:05.577 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj846 datafile 2023-12-01T06:42:05.745 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:05.745 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj847 datafile 2023-12-01T06:42:05.896 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:05.896 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj848 datafile 2023-12-01T06:42:06.080 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:06.081 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj849 datafile 2023-12-01T06:42:06.231 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:06.231 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj850 datafile 2023-12-01T06:42:06.391 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:06.391 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj851 datafile 2023-12-01T06:42:06.558 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:06.559 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj852 datafile 2023-12-01T06:42:06.718 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:06.718 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj853 datafile 2023-12-01T06:42:06.869 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:06.869 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj854 datafile 2023-12-01T06:42:07.036 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:07.036 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj855 datafile 2023-12-01T06:42:07.196 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:07.196 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj856 datafile 2023-12-01T06:42:07.379 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:07.379 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj857 datafile 2023-12-01T06:42:07.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:07.539 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj858 datafile 2023-12-01T06:42:07.691 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:07.691 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj859 datafile 2023-12-01T06:42:07.867 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:07.867 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj860 datafile 2023-12-01T06:42:08.009 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:08.009 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj861 datafile 2023-12-01T06:42:08.161 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:08.161 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj862 datafile 2023-12-01T06:42:08.320 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:08.320 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj863 datafile 2023-12-01T06:42:08.470 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:08.470 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj864 datafile 2023-12-01T06:42:08.621 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:08.621 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj865 datafile 2023-12-01T06:42:08.798 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:08.798 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj866 datafile 2023-12-01T06:42:08.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:08.948 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj867 datafile 2023-12-01T06:42:09.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:09.132 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj868 datafile 2023-12-01T06:42:09.297 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:09.297 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj869 datafile 2023-12-01T06:42:09.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:09.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj870 datafile 2023-12-01T06:42:09.627 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:09.628 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj871 datafile 2023-12-01T06:42:09.795 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:09.795 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj872 datafile 2023-12-01T06:42:10.015 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:10.015 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj873 datafile 2023-12-01T06:42:10.180 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:10.181 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj874 datafile 2023-12-01T06:42:10.332 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:10.333 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj875 datafile 2023-12-01T06:42:10.491 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:10.491 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj876 datafile 2023-12-01T06:42:10.651 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:10.651 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj877 datafile 2023-12-01T06:42:10.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:10.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj878 datafile 2023-12-01T06:42:10.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:10.987 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj879 datafile 2023-12-01T06:42:11.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:11.146 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj880 datafile 2023-12-01T06:42:11.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:11.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj881 datafile 2023-12-01T06:42:11.473 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:11.473 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj882 datafile 2023-12-01T06:42:11.771 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:11.771 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj883 datafile 2023-12-01T06:42:11.967 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:12.046 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj884 datafile 2023-12-01T06:42:12.143 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:12.143 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj885 datafile 2023-12-01T06:42:12.302 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:12.302 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj886 datafile 2023-12-01T06:42:12.452 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:12.453 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj887 datafile 2023-12-01T06:42:12.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:12.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj888 datafile 2023-12-01T06:42:12.746 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:12.747 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj889 datafile 2023-12-01T06:42:12.898 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:12.899 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj890 datafile 2023-12-01T06:42:13.073 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:13.074 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj891 datafile 2023-12-01T06:42:13.233 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:13.233 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj892 datafile 2023-12-01T06:42:13.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:13.409 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj893 datafile 2023-12-01T06:42:13.560 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:13.560 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj894 datafile 2023-12-01T06:42:13.718 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:13.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj895 datafile 2023-12-01T06:42:13.895 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:13.896 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj896 datafile 2023-12-01T06:42:14.055 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:14.055 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj897 datafile 2023-12-01T06:42:14.213 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:14.213 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj898 datafile 2023-12-01T06:42:14.373 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:14.373 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj899 datafile 2023-12-01T06:42:14.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:14.549 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj900 datafile 2023-12-01T06:42:14.709 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:14.710 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj901 datafile 2023-12-01T06:42:14.868 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:14.868 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj902 datafile 2023-12-01T06:42:15.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:15.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj903 datafile 2023-12-01T06:42:15.169 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:15.170 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj904 datafile 2023-12-01T06:42:15.329 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:15.329 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj905 datafile 2023-12-01T06:42:15.488 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:15.488 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj906 datafile 2023-12-01T06:42:15.639 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:15.639 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj907 datafile 2023-12-01T06:42:15.798 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:15.798 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj908 datafile 2023-12-01T06:42:16.005 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:16.005 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj909 datafile 2023-12-01T06:42:16.185 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:16.185 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj910 datafile 2023-12-01T06:42:16.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:16.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj911 datafile 2023-12-01T06:42:16.494 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:16.494 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj912 datafile 2023-12-01T06:42:16.645 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:16.645 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj913 datafile 2023-12-01T06:42:17.064 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:17.064 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj914 datafile 2023-12-01T06:42:17.215 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:17.215 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj915 datafile 2023-12-01T06:42:17.375 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:17.375 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj916 datafile 2023-12-01T06:42:17.550 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:17.550 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj917 datafile 2023-12-01T06:42:17.709 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:17.709 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj918 datafile 2023-12-01T06:42:17.869 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:17.870 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj919 datafile 2023-12-01T06:42:18.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:18.021 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj920 datafile 2023-12-01T06:42:18.178 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:18.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj921 datafile 2023-12-01T06:42:18.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:18.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj922 datafile 2023-12-01T06:42:18.489 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:18.489 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj923 datafile 2023-12-01T06:42:18.640 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:18.640 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj924 datafile 2023-12-01T06:42:18.782 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:18.783 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj925 datafile 2023-12-01T06:42:18.933 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:18.933 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj926 datafile 2023-12-01T06:42:19.118 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:19.118 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj927 datafile 2023-12-01T06:42:19.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:19.286 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj928 datafile 2023-12-01T06:42:19.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:19.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj929 datafile 2023-12-01T06:42:19.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:19.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj930 datafile 2023-12-01T06:42:19.764 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:19.764 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj931 datafile 2023-12-01T06:42:19.974 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:19.974 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj932 datafile 2023-12-01T06:42:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj933 datafile 2023-12-01T06:42:20.309 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:20.309 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj934 datafile 2023-12-01T06:42:20.468 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:20.468 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj935 datafile 2023-12-01T06:42:20.627 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:20.627 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj936 datafile 2023-12-01T06:42:20.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:20.787 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj937 datafile 2023-12-01T06:42:20.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:20.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj938 datafile 2023-12-01T06:42:21.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:21.130 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj939 datafile 2023-12-01T06:42:21.290 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:21.290 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj940 datafile 2023-12-01T06:42:21.457 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:21.457 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj941 datafile 2023-12-01T06:42:21.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:21.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj942 datafile 2023-12-01T06:42:21.759 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:21.759 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj943 datafile 2023-12-01T06:42:22.039 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:22.039 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj944 datafile 2023-12-01T06:42:22.237 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:22.237 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj945 datafile 2023-12-01T06:42:22.387 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:22.387 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj946 datafile 2023-12-01T06:42:22.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:22.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj947 datafile 2023-12-01T06:42:22.714 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:22.715 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj948 datafile 2023-12-01T06:42:22.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:22.890 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj949 datafile 2023-12-01T06:42:23.041 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:23.042 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj950 datafile 2023-12-01T06:42:23.201 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:23.201 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj951 datafile 2023-12-01T06:42:23.377 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:23.377 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj952 datafile 2023-12-01T06:42:23.536 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:23.536 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj953 datafile 2023-12-01T06:42:23.695 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:23.695 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj954 datafile 2023-12-01T06:42:23.863 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:23.863 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj955 datafile 2023-12-01T06:42:24.014 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:24.014 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj956 datafile 2023-12-01T06:42:24.165 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:24.166 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj957 datafile 2023-12-01T06:42:24.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:24.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj958 datafile 2023-12-01T06:42:24.466 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:24.466 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj959 datafile 2023-12-01T06:42:24.628 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:24.628 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj960 datafile 2023-12-01T06:42:24.779 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:24.780 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj961 datafile 2023-12-01T06:42:24.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:24.955 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj962 datafile 2023-12-01T06:42:25.111 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:25.111 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj963 datafile 2023-12-01T06:42:25.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:25.268 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj964 datafile 2023-12-01T06:42:25.441 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:25.441 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj965 datafile 2023-12-01T06:42:25.583 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:25.583 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj966 datafile 2023-12-01T06:42:25.759 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:25.760 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj967 datafile 2023-12-01T06:42:25.937 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:25.937 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj968 datafile 2023-12-01T06:42:26.094 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:26.094 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj969 datafile 2023-12-01T06:42:26.240 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:26.240 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj970 datafile 2023-12-01T06:42:26.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:26.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj971 datafile 2023-12-01T06:42:26.565 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:26.566 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj972 datafile 2023-12-01T06:42:26.725 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:26.725 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj973 datafile 2023-12-01T06:42:26.884 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:26.884 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj974 datafile 2023-12-01T06:42:27.043 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:27.043 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj975 datafile 2023-12-01T06:42:27.252 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:27.252 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj976 datafile 2023-12-01T06:42:27.430 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:27.431 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj977 datafile 2023-12-01T06:42:27.571 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:27.571 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj978 datafile 2023-12-01T06:42:27.730 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:27.730 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj979 datafile 2023-12-01T06:42:27.907 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:27.907 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj980 datafile 2023-12-01T06:42:28.083 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:28.084 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj981 datafile 2023-12-01T06:42:28.251 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:28.251 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj982 datafile 2023-12-01T06:42:28.402 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:28.402 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj983 datafile 2023-12-01T06:42:28.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:28.603 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj984 datafile 2023-12-01T06:42:28.753 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:28.754 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj985 datafile 2023-12-01T06:42:28.913 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:28.913 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj986 datafile 2023-12-01T06:42:29.081 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:29.081 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj987 datafile 2023-12-01T06:42:29.248 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:29.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj988 datafile 2023-12-01T06:42:29.433 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:29.433 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj989 datafile 2023-12-01T06:42:29.583 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:29.583 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj990 datafile 2023-12-01T06:42:29.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:29.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj991 datafile 2023-12-01T06:42:29.919 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:29.919 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj992 datafile 2023-12-01T06:42:30.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:30.071 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj993 datafile 2023-12-01T06:42:30.229 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:30.230 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj994 datafile 2023-12-01T06:42:30.373 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:30.373 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj995 datafile 2023-12-01T06:42:30.540 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:30.540 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj996 datafile 2023-12-01T06:42:30.699 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:30.699 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj997 datafile 2023-12-01T06:42:30.875 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:30.875 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj998 datafile 2023-12-01T06:42:31.064 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:31.064 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj999 datafile 2023-12-01T06:42:31.243 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:76: TEST_recover_unexpected: for j in $(seq 1 $OBJS) 2023-12-01T06:42:31.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:78: TEST_recover_unexpected: rados -p test3 put obj1000 datafile 2023-12-01T06:42:31.479 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:81: TEST_recover_unexpected: rm datafile 2023-12-01T06:42:31.481 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:83: TEST_recover_unexpected: ceph osd set noscrub 2023-12-01T06:42:31.976 INFO:tasks.workunit.client.0.smithi170.stderr:noscrub is set 2023-12-01T06:42:31.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:84: TEST_recover_unexpected: ceph osd set nodeep-scrub 2023-12-01T06:42:32.512 INFO:tasks.workunit.client.0.smithi170.stderr:nodeep-scrub is set 2023-12-01T06:42:32.535 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: ceph pg dump pgs --format=json-pretty 2023-12-01T06:42:32.535 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: jq '.pg_stats[].pgid' 2023-12-01T06:42:32.853 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:42:32.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:32.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.9"' 2023-12-01T06:42:32.871 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.9 2023-12-01T06:42:32.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.9 scrub 2023-12-01T06:42:33.017 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:33.018 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:33.018 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:33.018 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:33.018 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:33.033 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:33.033 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.b"' 2023-12-01T06:42:33.033 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.b 2023-12-01T06:42:33.033 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.b scrub 2023-12-01T06:42:33.176 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:33.176 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:33.176 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:33.176 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:33.177 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:33.191 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:33.191 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.8"' 2023-12-01T06:42:33.191 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.8 2023-12-01T06:42:33.191 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.8 scrub 2023-12-01T06:42:33.345 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:33.345 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:33.345 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:33.345 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:33.345 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:33.360 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:33.360 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.8"' 2023-12-01T06:42:33.360 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.8 2023-12-01T06:42:33.360 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.8 scrub 2023-12-01T06:42:33.501 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:33.501 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:33.501 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:33.501 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:33.502 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:33.516 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:33.516 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.a"' 2023-12-01T06:42:33.516 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.a 2023-12-01T06:42:33.516 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.a scrub 2023-12-01T06:42:33.657 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:33.657 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:33.657 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:33.657 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:33.657 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:33.672 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:33.672 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.9"' 2023-12-01T06:42:33.672 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.9 2023-12-01T06:42:33.672 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.9 scrub 2023-12-01T06:42:33.821 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:33.821 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:33.821 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:33.821 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:33.821 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:33.836 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:33.836 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.b"' 2023-12-01T06:42:33.836 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.b 2023-12-01T06:42:33.836 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.b scrub 2023-12-01T06:42:33.981 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:33.981 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:33.982 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:33.982 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:33.982 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:33.996 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:33.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.9"' 2023-12-01T06:42:33.997 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.9 2023-12-01T06:42:33.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.9 scrub 2023-12-01T06:42:34.137 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:34.137 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:34.138 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:34.138 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:34.138 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:34.153 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:34.153 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.a"' 2023-12-01T06:42:34.153 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.a 2023-12-01T06:42:34.153 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.a scrub 2023-12-01T06:42:34.290 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:34.291 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:34.291 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:34.291 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:34.291 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:34.304 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:34.305 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.a"' 2023-12-01T06:42:34.305 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.a 2023-12-01T06:42:34.305 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.a scrub 2023-12-01T06:42:34.443 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:34.443 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:34.443 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:34.444 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:34.444 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:34.458 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:34.458 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.8"' 2023-12-01T06:42:34.458 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.8 2023-12-01T06:42:34.458 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.8 scrub 2023-12-01T06:42:34.591 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:34.592 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:34.592 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:34.592 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:34.592 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:34.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:34.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.b"' 2023-12-01T06:42:34.605 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.b 2023-12-01T06:42:34.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.b scrub 2023-12-01T06:42:34.748 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:34.748 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:34.748 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:34.748 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:34.748 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:34.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:34.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.5"' 2023-12-01T06:42:34.762 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.5 2023-12-01T06:42:34.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.5 scrub 2023-12-01T06:42:34.896 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:34.896 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:34.896 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:34.896 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:34.896 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:34.909 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:34.909 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.7"' 2023-12-01T06:42:34.910 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.7 2023-12-01T06:42:34.910 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.7 scrub 2023-12-01T06:42:35.045 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:35.045 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:35.045 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:35.046 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:35.046 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:35.059 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:35.059 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.4"' 2023-12-01T06:42:35.060 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.4 2023-12-01T06:42:35.060 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.4 scrub 2023-12-01T06:42:35.201 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:35.201 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:35.201 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:35.201 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:35.201 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:35.215 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:35.215 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.4"' 2023-12-01T06:42:35.215 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.4 2023-12-01T06:42:35.216 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.4 scrub 2023-12-01T06:42:35.356 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:35.356 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:35.356 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:35.356 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:35.356 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:35.370 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:35.370 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.6"' 2023-12-01T06:42:35.370 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.6 2023-12-01T06:42:35.370 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.6 scrub 2023-12-01T06:42:35.514 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:35.514 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:35.514 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:35.515 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:35.515 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:35.526 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:35.527 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.5"' 2023-12-01T06:42:35.527 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.5 2023-12-01T06:42:35.527 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.5 scrub 2023-12-01T06:42:35.669 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:35.669 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:35.669 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:35.669 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:35.669 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:35.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:35.683 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.7"' 2023-12-01T06:42:35.684 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.7 2023-12-01T06:42:35.684 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.7 scrub 2023-12-01T06:42:35.824 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:35.824 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:35.824 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:35.824 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:35.824 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:35.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:35.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.5"' 2023-12-01T06:42:35.839 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.5 2023-12-01T06:42:35.839 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.5 scrub 2023-12-01T06:42:35.972 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:35.972 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:35.972 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:35.972 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:35.972 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:35.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:35.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.6"' 2023-12-01T06:42:35.986 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.6 2023-12-01T06:42:35.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.6 scrub 2023-12-01T06:42:36.131 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:36.131 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:36.131 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:36.131 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:36.131 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:36.145 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:36.145 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.6"' 2023-12-01T06:42:36.145 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.6 2023-12-01T06:42:36.145 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.6 scrub 2023-12-01T06:42:36.287 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:36.287 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:36.287 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:36.287 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:36.287 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:36.300 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:36.301 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.4"' 2023-12-01T06:42:36.301 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.4 2023-12-01T06:42:36.301 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.4 scrub 2023-12-01T06:42:36.440 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:36.440 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:36.440 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:36.440 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:36.440 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:36.455 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:36.455 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.7"' 2023-12-01T06:42:36.455 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.7 2023-12-01T06:42:36.455 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.7 scrub 2023-12-01T06:42:36.596 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:36.596 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:36.596 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:36.596 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:36.596 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:36.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:36.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.1"' 2023-12-01T06:42:36.610 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.1 2023-12-01T06:42:36.611 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.1 scrub 2023-12-01T06:42:36.751 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:36.751 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:36.751 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:36.751 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:36.751 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:36.764 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:36.764 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.3"' 2023-12-01T06:42:36.764 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.3 2023-12-01T06:42:36.764 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.3 scrub 2023-12-01T06:42:36.894 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:36.894 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:36.894 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:36.894 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:36.895 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:36.907 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:36.907 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.0"' 2023-12-01T06:42:36.907 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.0 2023-12-01T06:42:36.907 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.0 scrub 2023-12-01T06:42:37.049 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:37.049 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:37.049 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:37.049 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:37.049 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:37.067 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:37.067 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.0"' 2023-12-01T06:42:37.067 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.0 2023-12-01T06:42:37.067 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.0 scrub 2023-12-01T06:42:37.204 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:37.204 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:37.204 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:37.204 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:37.205 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:37.218 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:37.218 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.2"' 2023-12-01T06:42:37.219 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.2 2023-12-01T06:42:37.219 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.2 scrub 2023-12-01T06:42:37.385 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:37.385 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:37.385 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:37.385 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:37.385 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:37.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:37.399 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.1"' 2023-12-01T06:42:37.399 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.1 2023-12-01T06:42:37.400 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.1 scrub 2023-12-01T06:42:37.533 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:37.533 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:37.533 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:37.533 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:37.533 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:37.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:37.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.e"' 2023-12-01T06:42:37.547 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.e 2023-12-01T06:42:37.547 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.e scrub 2023-12-01T06:42:37.685 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:37.685 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:37.685 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:37.685 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:37.686 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:37.699 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:37.699 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.f"' 2023-12-01T06:42:37.699 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.f 2023-12-01T06:42:37.699 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.f scrub 2023-12-01T06:42:37.838 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:37.838 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:37.838 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:37.838 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:37.838 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:37.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:37.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.c"' 2023-12-01T06:42:37.851 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.c 2023-12-01T06:42:37.852 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.c scrub 2023-12-01T06:42:37.991 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:37.991 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:37.991 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:37.991 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:37.992 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:38.005 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:38.005 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.3"' 2023-12-01T06:42:38.006 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.3 2023-12-01T06:42:38.006 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.3 scrub 2023-12-01T06:42:38.138 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:38.139 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:38.139 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:38.139 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:38.139 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:38.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:38.152 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.1"' 2023-12-01T06:42:38.152 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.1 2023-12-01T06:42:38.153 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.1 scrub 2023-12-01T06:42:38.292 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:38.292 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:38.292 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:38.292 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:38.293 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:38.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:38.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.2"' 2023-12-01T06:42:38.307 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.2 2023-12-01T06:42:38.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.2 scrub 2023-12-01T06:42:38.451 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:38.451 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:38.451 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:38.451 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:38.451 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:38.465 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:38.466 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.2"' 2023-12-01T06:42:38.466 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.2 2023-12-01T06:42:38.466 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.2 scrub 2023-12-01T06:42:38.603 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:38.604 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:38.604 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:38.604 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:38.604 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:38.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:38.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.0"' 2023-12-01T06:42:38.618 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.0 2023-12-01T06:42:38.618 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.0 scrub 2023-12-01T06:42:38.766 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:38.766 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:38.766 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:38.766 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:38.766 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:38.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:38.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.3"' 2023-12-01T06:42:38.781 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.3 2023-12-01T06:42:38.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.3 scrub 2023-12-01T06:42:38.921 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:38.921 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:38.921 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:38.921 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:38.921 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:38.936 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:38.936 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.f"' 2023-12-01T06:42:38.936 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.f 2023-12-01T06:42:38.936 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.f scrub 2023-12-01T06:42:39.082 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:39.083 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:39.083 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:39.083 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:39.083 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:39.097 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:39.097 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.e"' 2023-12-01T06:42:39.097 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.e 2023-12-01T06:42:39.098 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.e scrub 2023-12-01T06:42:39.236 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:39.237 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:39.237 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:39.237 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:39.237 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:39.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:39.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.d"' 2023-12-01T06:42:39.251 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.d 2023-12-01T06:42:39.251 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.d scrub 2023-12-01T06:42:39.393 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:39.393 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:39.393 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:39.393 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:39.394 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:39.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:39.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.c"' 2023-12-01T06:42:39.408 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.c 2023-12-01T06:42:39.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.c scrub 2023-12-01T06:42:39.538 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:39.538 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:39.538 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:39.538 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:39.538 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:39.552 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:39.552 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.d"' 2023-12-01T06:42:39.552 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.d 2023-12-01T06:42:39.552 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.d scrub 2023-12-01T06:42:39.687 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:39.687 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:39.687 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:39.688 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:39.688 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:39.702 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:39.702 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.e"' 2023-12-01T06:42:39.702 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.e 2023-12-01T06:42:39.702 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.e scrub 2023-12-01T06:42:39.845 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:39.845 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:39.845 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:39.845 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:39.845 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:39.859 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:39.859 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="3.d"' 2023-12-01T06:42:39.859 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=3.d 2023-12-01T06:42:39.860 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 3.d scrub 2023-12-01T06:42:39.994 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:39.995 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:39.995 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:39.995 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:39.995 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:40.008 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:40.008 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="1.f"' 2023-12-01T06:42:40.008 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=1.f 2023-12-01T06:42:40.008 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 1.f scrub 2023-12-01T06:42:40.144 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:40.144 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:40.144 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:40.144 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:40.144 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:40.157 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:86: TEST_recover_unexpected: for qpg in $(ceph pg dump pgs --format=json-pretty | jq '.pg_stats[].pgid') 2023-12-01T06:42:40.157 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: eval 'pg="2.c"' 2023-12-01T06:42:40.157 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:88: TEST_recover_unexpected: pg=2.c 2023-12-01T06:42:40.157 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:89: TEST_recover_unexpected: ceph tell 2.c scrub 2023-12-01T06:42:40.295 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:40.295 INFO:tasks.workunit.client.0.smithi170.stdout: "deep": false, 2023-12-01T06:42:40.295 INFO:tasks.workunit.client.0.smithi170.stdout: "must": true, 2023-12-01T06:42:40.295 INFO:tasks.workunit.client.0.smithi170.stdout: "stamp": "0.000000" 2023-12-01T06:42:40.295 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:40.308 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:92: TEST_recover_unexpected: ceph pg dump pgs 2023-12-01T06:42:40.605 INFO:tasks.workunit.client.0.smithi170.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2023-12-01T06:42:40.606 INFO:tasks.workunit.client.0.smithi170.stdout:3.9 61 0 0 0 0 499712 0 0 61 0 61 active+clean+scrubbing 2023-12-01T06:42:33.632021+0000 43'61 45:86 [4,2,5] 4 [4,2,5] 4 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 scrubbing for 4s 50 0 2023-12-01T06:42:40.606 INFO:tasks.workunit.client.0.smithi170.stdout:1.b 65 0 0 0 0 532480 0 0 65 0 65 active+clean 2023-12-01T06:34:06.948941+0000 43'65 43:90 [1,0,2] 1 [1,0,2] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.606 INFO:tasks.workunit.client.0.smithi170.stdout:2.8 66 0 0 0 0 540672 0 0 66 0 66 active+clean+scrubbing 2023-12-01T06:42:33.880818+0000 43'66 45:94 [0,2,1] 0 [0,2,1] 0 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 scrubbing for 3s 50 0 2023-12-01T06:42:40.606 INFO:tasks.workunit.client.0.smithi170.stdout:3.8 66 0 0 0 0 540672 0 0 66 0 66 active+clean 2023-12-01T06:34:10.855897+0000 43'66 45:86 [4,1,2] 4 [4,1,2] 4 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.606 INFO:tasks.workunit.client.0.smithi170.stdout:1.a 61 0 0 0 0 499712 0 0 61 0 61 active+clean 2023-12-01T06:34:06.950107+0000 43'61 45:86 [1,2,4] 1 [1,2,4] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.606 INFO:tasks.workunit.client.0.smithi170.stdout:2.9 61 0 0 0 0 499712 0 0 61 0 61 active+clean 2023-12-01T06:34:08.613968+0000 43'61 45:84 [4,1,0] 4 [4,1,0] 4 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.606 INFO:tasks.workunit.client.0.smithi170.stdout:3.b 65 0 0 0 0 532480 0 0 65 0 65 active+clean 2023-12-01T06:34:10.883823+0000 43'65 45:84 [3,0,4] 3 [3,0,4] 3 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.606 INFO:tasks.workunit.client.0.smithi170.stdout:1.9 61 0 0 0 0 499712 0 0 61 0 61 active+clean 2023-12-01T06:34:06.907969+0000 43'61 45:86 [1,4,5] 1 [1,4,5] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.606 INFO:tasks.workunit.client.0.smithi170.stdout:2.a 61 0 0 0 0 499712 0 0 61 0 61 active+clean 2023-12-01T06:34:08.656224+0000 43'61 45:83 [1,5,0] 1 [1,5,0] 1 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.606 INFO:tasks.workunit.client.0.smithi170.stdout:3.a 61 0 0 0 0 499712 0 0 61 0 61 active+clean 2023-12-01T06:34:10.565046+0000 43'61 45:80 [3,1,5] 3 [3,1,5] 3 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.606 INFO:tasks.workunit.client.0.smithi170.stdout:1.8 66 0 0 0 0 540672 0 0 66 0 66 active+clean 2023-12-01T06:34:06.993467+0000 43'66 45:88 [5,4,3] 5 [5,4,3] 5 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.606 INFO:tasks.workunit.client.0.smithi170.stdout:2.b 65 0 0 0 0 532480 0 0 65 0 65 active+clean 2023-12-01T06:34:08.653399+0000 43'65 45:87 [3,2,5] 3 [3,2,5] 3 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.606 INFO:tasks.workunit.client.0.smithi170.stdout:3.5 73 0 0 0 0 598016 0 0 73 0 73 active+clean 2023-12-01T06:34:10.562840+0000 43'73 45:92 [5,3,4] 5 [5,3,4] 5 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.607 INFO:tasks.workunit.client.0.smithi170.stdout:1.7 53 0 0 0 0 434176 0 0 53 0 53 active+clean 2023-12-01T06:34:06.908569+0000 43'53 45:78 [1,3,0] 1 [1,3,0] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.607 INFO:tasks.workunit.client.0.smithi170.stdout:2.4 51 0 0 0 0 417792 0 0 51 0 51 active+clean 2023-12-01T06:34:08.651258+0000 43'51 45:74 [1,0,2] 1 [1,0,2] 1 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.607 INFO:tasks.workunit.client.0.smithi170.stdout:3.4 51 0 0 0 0 417792 0 0 51 0 51 active+clean 2023-12-01T06:34:10.855241+0000 43'51 45:71 [1,2,4] 1 [1,2,4] 1 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.607 INFO:tasks.workunit.client.0.smithi170.stdout:1.6 64 0 0 0 0 524288 0 0 64 0 64 active+clean 2023-12-01T06:34:06.992423+0000 43'64 43:87 [3,4,2] 3 [3,4,2] 3 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 periodic scrub scheduled @ 2023-12-02T12:39:11.317756+0000 0 0 2023-12-01T06:42:40.607 INFO:tasks.workunit.client.0.smithi170.stdout:2.5 73 0 0 0 0 598016 0 0 73 0 73 active+clean 2023-12-01T06:34:08.650539+0000 43'73 43:95 [3,4,0] 3 [3,4,0] 3 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 periodic scrub scheduled @ 2023-12-02T09:45:42.873806+0000 0 0 2023-12-01T06:42:40.607 INFO:tasks.workunit.client.0.smithi170.stdout:3.7 53 0 0 0 0 434176 0 0 53 0 53 active+clean 2023-12-01T06:34:10.714945+0000 43'53 45:73 [4,3,0] 4 [4,3,0] 4 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.607 INFO:tasks.workunit.client.0.smithi170.stdout:1.5 73 0 0 0 0 598016 0 0 73 0 73 active+clean+scrubbing 2023-12-01T06:42:36.705437+0000 43'73 45:103 [4,2,5] 4 [4,2,5] 4 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 scrubbing for 0s 0 0 2023-12-01T06:42:40.607 INFO:tasks.workunit.client.0.smithi170.stdout:2.6 64 0 0 0 0 524288 0 0 64 0 64 active+clean 2023-12-01T06:34:08.616014+0000 43'64 43:85 [1,0,4] 1 [1,0,4] 1 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 periodic scrub scheduled @ 2023-12-02T11:50:36.875829+0000 0 0 2023-12-01T06:42:40.607 INFO:tasks.workunit.client.0.smithi170.stdout:3.6 64 0 0 0 0 524288 0 0 64 0 64 active+clean 2023-12-01T06:34:10.999966+0000 43'64 45:83 [0,1,4] 0 [0,1,4] 0 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.607 INFO:tasks.workunit.client.0.smithi170.stdout:1.4 51 0 0 0 0 417792 0 0 51 0 51 active+clean 2023-12-01T06:34:07.048410+0000 43'51 43:75 [3,0,1] 3 [3,0,1] 3 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 periodic scrub scheduled @ 2023-12-02T12:03:35.455170+0000 0 0 2023-12-01T06:42:40.607 INFO:tasks.workunit.client.0.smithi170.stdout:2.7 53 0 0 0 0 434176 0 0 53 0 53 active+clean 2023-12-01T06:34:08.650762+0000 43'53 43:74 [3,0,2] 3 [3,0,2] 3 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 periodic scrub scheduled @ 2023-12-02T11:27:40.763457+0000 0 0 2023-12-01T06:42:40.607 INFO:tasks.workunit.client.0.smithi170.stdout:3.1 67 0 0 0 0 548864 0 0 67 0 67 active+clean 2023-12-01T06:34:10.563124+0000 43'67 45:88 [4,3,5] 4 [4,3,5] 4 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.607 INFO:tasks.workunit.client.0.smithi170.stdout:1.3 54 0 0 0 0 442368 0 0 54 0 54 active+clean 2023-12-01T06:34:06.991139+0000 43'54 45:81 [4,1,2] 4 [4,1,2] 4 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.607 INFO:tasks.workunit.client.0.smithi170.stdout:2.0 72 0 0 0 0 589824 0 0 72 0 72 active+clean 2023-12-01T06:34:08.652076+0000 43'72 43:94 [3,5,1] 3 [3,5,1] 3 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 periodic scrub scheduled @ 2023-12-02T10:56:31.712258+0000 0 0 2023-12-01T06:42:40.608 INFO:tasks.workunit.client.0.smithi170.stdout:3.0 72 0 0 0 0 589824 0 0 72 0 72 active+clean 2023-12-01T06:34:10.854449+0000 43'72 43:91 [1,2,4] 1 [1,2,4] 1 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 periodic scrub scheduled @ 2023-12-02T15:16:58.646857+0000 0 0 2023-12-01T06:42:40.608 INFO:tasks.workunit.client.0.smithi170.stdout:1.2 55 0 0 0 0 450560 0 0 55 0 55 active+clean 2023-12-01T06:34:07.047458+0000 43'55 45:82 [0,4,1] 0 [0,4,1] 0 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.608 INFO:tasks.workunit.client.0.smithi170.stdout:2.1 67 0 0 0 0 548864 0 0 67 0 67 active+clean+scrubbing 2023-12-01T06:42:38.293167+0000 43'67 45:91 [2,5,0] 2 [2,5,0] 2 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 scrubbing for 0s 0 0 2023-12-01T06:42:40.608 INFO:tasks.workunit.client.0.smithi170.stdout:3.e 69 0 0 0 0 565248 0 0 69 0 69 active+clean 2023-12-01T06:34:10.999747+0000 43'69 45:87 [2,4,1] 2 [2,4,1] 2 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.608 INFO:tasks.workunit.client.0.smithi170.stdout:2.f 62 0 0 0 0 507904 0 0 62 0 62 active+clean 2023-12-01T06:34:08.509209+0000 43'62 45:85 [0,5,4] 0 [0,5,4] 0 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:40.608 INFO:tasks.workunit.client.0.smithi170.stdout:1.c 60 0 0 0 0 491520 0 0 60 0 60 active+clean 2023-12-01T06:34:07.050262+0000 43'60 43:83 [3,0,4] 3 [3,0,4] 3 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 periodic scrub scheduled @ 2023-12-02T11:22:04.311247+0000 0 0 2023-12-01T06:42:40.608 INFO:tasks.workunit.client.0.smithi170.stdout:3.3 54 0 0 0 0 442368 0 0 54 0 54 active+clean 2023-12-01T06:34:10.714920+0000 43'54 44:74 [4,0,5] 4 [4,0,5] 4 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 periodic scrub scheduled @ 2023-12-02T07:27:01.574922+0000 0 0 2023-12-01T06:42:40.608 INFO:tasks.workunit.client.0.smithi170.stdout:1.1 67 0 0 0 0 548864 0 0 67 0 67 active+clean 2023-12-01T06:34:07.048236+0000 43'67 43:89 [5,0,4] 5 [5,0,4] 5 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 periodic scrub scheduled @ 2023-12-02T06:39:02.205846+0000 0 0 2023-12-01T06:42:40.608 INFO:tasks.workunit.client.0.smithi170.stdout:2.2 55 0 0 0 0 450560 0 0 55 0 55 active+clean 2023-12-01T06:34:08.649704+0000 43'55 43:75 [5,1,0] 5 [5,1,0] 5 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 periodic scrub scheduled @ 2023-12-02T13:02:55.810152+0000 0 0 2023-12-01T06:42:40.608 INFO:tasks.workunit.client.0.smithi170.stdout:3.2 55 0 0 0 0 450560 0 0 55 0 55 active+clean 2023-12-01T06:34:10.884368+0000 43'55 43:74 [3,5,0] 3 [3,5,0] 3 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 periodic scrub scheduled @ 2023-12-02T10:05:27.728253+0000 0 0 2023-12-01T06:42:40.608 INFO:tasks.workunit.client.0.smithi170.stdout:1.0 72 0 0 0 0 589824 0 0 72 0 72 active+clean 2023-12-01T06:34:06.949811+0000 43'72 43:97 [1,0,2] 1 [1,0,2] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 periodic scrub scheduled @ 2023-12-02T12:16:28.122534+0000 0 0 2023-12-01T06:42:40.608 INFO:tasks.workunit.client.0.smithi170.stdout:2.3 54 0 0 0 0 442368 0 0 54 0 54 active+clean 2023-12-01T06:34:08.651725+0000 43'54 43:74 [5,4,1] 5 [5,4,1] 5 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 periodic scrub scheduled @ 2023-12-02T08:56:56.113095+0000 0 0 2023-12-01T06:42:40.608 INFO:tasks.workunit.client.0.smithi170.stdout:3.f 62 0 0 0 0 507904 0 0 62 0 62 active+clean 2023-12-01T06:34:10.999133+0000 43'62 43:80 [0,1,4] 0 [0,1,4] 0 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 periodic scrub scheduled @ 2023-12-02T08:16:03.451443+0000 0 0 2023-12-01T06:42:40.608 INFO:tasks.workunit.client.0.smithi170.stdout:2.e 69 0 0 0 0 565248 0 0 69 0 69 active+clean 2023-12-01T06:34:08.648989+0000 43'69 43:88 [2,3,5] 2 [2,3,5] 2 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 periodic scrub scheduled @ 2023-12-02T17:55:48.560107+0000 0 0 2023-12-01T06:42:40.609 INFO:tasks.workunit.client.0.smithi170.stdout:1.d 67 0 0 0 0 548864 0 0 67 0 67 active+clean 2023-12-01T06:34:06.994921+0000 43'67 43:88 [2,4,1] 2 [2,4,1] 2 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 periodic scrub scheduled @ 2023-12-02T08:55:33.837077+0000 0 0 2023-12-01T06:42:40.609 INFO:tasks.workunit.client.0.smithi170.stdout:3.c 60 0 0 0 0 491520 0 0 60 0 60 active+clean 2023-12-01T06:34:10.562236+0000 43'60 43:77 [5,3,1] 5 [5,3,1] 5 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 periodic scrub scheduled @ 2023-12-02T12:25:09.002008+0000 0 0 2023-12-01T06:42:40.609 INFO:tasks.workunit.client.0.smithi170.stdout:2.d 67 0 0 0 0 548864 0 0 67 0 67 active+clean 2023-12-01T06:34:08.613058+0000 43'67 43:86 [5,1,3] 5 [5,1,3] 5 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 periodic scrub scheduled @ 2023-12-02T07:10:51.979255+0000 0 0 2023-12-01T06:42:40.609 INFO:tasks.workunit.client.0.smithi170.stdout:1.e 69 0 0 0 0 565248 0 0 69 0 69 active+clean 2023-12-01T06:34:06.950499+0000 43'69 43:93 [1,4,2] 1 [1,4,2] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 periodic scrub scheduled @ 2023-12-02T18:12:34.331678+0000 0 0 2023-12-01T06:42:40.609 INFO:tasks.workunit.client.0.smithi170.stdout:3.d 67 0 0 0 0 548864 0 0 67 0 67 active+clean 2023-12-01T06:34:10.883539+0000 43'67 43:85 [3,1,0] 3 [3,1,0] 3 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 periodic scrub scheduled @ 2023-12-02T08:40:16.346842+0000 0 0 2023-12-01T06:42:40.609 INFO:tasks.workunit.client.0.smithi170.stdout:1.f 62 0 0 0 0 507904 0 0 62 0 62 active+clean 2023-12-01T06:34:07.048928+0000 43'62 43:85 [3,0,4] 3 [3,0,4] 3 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 periodic scrub scheduled @ 2023-12-02T10:42:53.215570+0000 0 0 2023-12-01T06:42:40.609 INFO:tasks.workunit.client.0.smithi170.stdout:2.c 60 0 0 0 0 491520 0 0 60 0 60 active+clean 2023-12-01T06:34:08.651633+0000 43'60 43:79 [2,0,5] 2 [2,0,5] 2 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 periodic scrub scheduled @ 2023-12-02T13:47:15.084685+0000 0 0 2023-12-01T06:42:40.609 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:42:40.609 INFO:tasks.workunit.client.0.smithi170.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2023-12-01T06:42:40.609 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:42:40.624 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:94: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:40.624 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:94: TEST_recover_unexpected: jq .osd_max_scrubs 2023-12-01T06:42:40.624 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:40.624 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:40.624 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:40.625 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:40.625 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:40.625 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:40.625 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:94: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:40.625 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:94: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:40.740 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:94: TEST_recover_unexpected: max=4 2023-12-01T06:42:40.740 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:95: TEST_recover_unexpected: '[' 4 '!=' 4 ']' 2023-12-01T06:42:40.740 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:100: TEST_recover_unexpected: ceph osd unset noscrub 2023-12-01T06:42:41.160 INFO:tasks.workunit.client.0.smithi170.stderr:noscrub is unset 2023-12-01T06:42:41.179 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:102: TEST_recover_unexpected: ok=false 2023-12-01T06:42:41.180 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:103: TEST_recover_unexpected: seq 0 300 2023-12-01T06:42:41.181 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:103: TEST_recover_unexpected: for i in $(seq 0 300) 2023-12-01T06:42:41.181 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:105: TEST_recover_unexpected: ceph pg dump pgs 2023-12-01T06:42:41.490 INFO:tasks.workunit.client.0.smithi170.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2023-12-01T06:42:41.491 INFO:tasks.workunit.client.0.smithi170.stdout:3.9 61 0 0 0 0 499712 0 0 61 0 61 active+clean+scrubbing 2023-12-01T06:42:33.632021+0000 43'61 45:86 [4,2,5] 4 [4,2,5] 4 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 scrubbing for 4s 50 0 2023-12-01T06:42:41.491 INFO:tasks.workunit.client.0.smithi170.stdout:1.b 65 0 0 0 0 532480 0 0 65 0 65 active+clean+scrubbing 2023-12-01T06:42:38.485218+0000 43'65 45:94 [1,0,2] 1 [1,0,2] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 scrubbing for 2s 0 0 2023-12-01T06:42:41.491 INFO:tasks.workunit.client.0.smithi170.stdout:2.8 66 0 0 0 0 540672 0 0 66 0 66 active+clean+scrubbing 2023-12-01T06:42:33.880818+0000 43'66 45:94 [0,2,1] 0 [0,2,1] 0 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 scrubbing for 3s 50 0 2023-12-01T06:42:41.491 INFO:tasks.workunit.client.0.smithi170.stdout:3.8 66 0 0 0 0 540672 0 0 66 0 66 active+clean 2023-12-01T06:34:10.855897+0000 43'66 45:86 [4,1,2] 4 [4,1,2] 4 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.491 INFO:tasks.workunit.client.0.smithi170.stdout:1.a 61 0 0 0 0 499712 0 0 61 0 61 active+clean 2023-12-01T06:34:06.950107+0000 43'61 45:86 [1,2,4] 1 [1,2,4] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.491 INFO:tasks.workunit.client.0.smithi170.stdout:2.9 61 0 0 0 0 499712 0 0 61 0 61 active+clean 2023-12-01T06:34:08.613968+0000 43'61 45:84 [4,1,0] 4 [4,1,0] 4 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.491 INFO:tasks.workunit.client.0.smithi170.stdout:3.b 65 0 0 0 0 532480 0 0 65 0 65 active+clean 2023-12-01T06:34:10.883823+0000 43'65 45:84 [3,0,4] 3 [3,0,4] 3 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.491 INFO:tasks.workunit.client.0.smithi170.stdout:1.9 61 0 0 0 0 499712 0 0 61 0 61 active+clean+scrubbing 2023-12-01T06:42:39.526485+0000 43'61 45:90 [1,4,5] 1 [1,4,5] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 scrubbing for 1s 0 0 2023-12-01T06:42:41.491 INFO:tasks.workunit.client.0.smithi170.stdout:2.a 61 0 0 0 0 499712 0 0 61 0 61 active+clean 2023-12-01T06:34:08.656224+0000 43'61 45:83 [1,5,0] 1 [1,5,0] 1 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.491 INFO:tasks.workunit.client.0.smithi170.stdout:3.a 61 0 0 0 0 499712 0 0 61 0 61 active+clean 2023-12-01T06:34:10.565046+0000 43'61 45:80 [3,1,5] 3 [3,1,5] 3 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.492 INFO:tasks.workunit.client.0.smithi170.stdout:1.8 66 0 0 0 0 540672 0 0 66 0 66 active+clean+scrubbing 2023-12-01T06:42:35.637257+0000 43'66 45:93 [5,4,3] 5 [5,4,3] 5 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 scrubbing for 4s 50 0 2023-12-01T06:42:41.492 INFO:tasks.workunit.client.0.smithi170.stdout:2.b 65 0 0 0 0 532480 0 0 65 0 65 active+clean 2023-12-01T06:34:08.653399+0000 43'65 45:87 [3,2,5] 3 [3,2,5] 3 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.492 INFO:tasks.workunit.client.0.smithi170.stdout:3.5 73 0 0 0 0 598016 0 0 73 0 73 active+clean+scrubbing 2023-12-01T06:42:37.571464+0000 43'73 45:95 [5,3,4] 5 [5,3,4] 5 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 scrubbing for 0s 0 0 2023-12-01T06:42:41.492 INFO:tasks.workunit.client.0.smithi170.stdout:1.7 53 0 0 0 0 434176 0 0 53 0 53 active+clean+scrubbing 2023-12-01T06:42:37.510163+0000 43'53 45:83 [1,3,0] 1 [1,3,0] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 scrubbing for 3s 50 0 2023-12-01T06:42:41.492 INFO:tasks.workunit.client.0.smithi170.stdout:2.4 51 0 0 0 0 417792 0 0 51 0 51 active+clean 2023-12-01T06:34:08.651258+0000 43'51 45:74 [1,0,2] 1 [1,0,2] 1 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.492 INFO:tasks.workunit.client.0.smithi170.stdout:3.4 51 0 0 0 0 417792 0 0 51 0 51 active+clean 2023-12-01T06:34:10.855241+0000 43'51 45:71 [1,2,4] 1 [1,2,4] 1 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.492 INFO:tasks.workunit.client.0.smithi170.stdout:1.6 64 0 0 0 0 524288 0 0 64 0 64 active+clean+scrubbing 2023-12-01T06:42:39.493151+0000 43'64 45:92 [3,4,2] 3 [3,4,2] 3 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 scrubbing for 0s 0 0 2023-12-01T06:42:41.492 INFO:tasks.workunit.client.0.smithi170.stdout:2.5 73 0 0 0 0 598016 0 0 73 0 73 active+clean 2023-12-01T06:34:08.650539+0000 43'73 45:96 [3,4,0] 3 [3,4,0] 3 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.492 INFO:tasks.workunit.client.0.smithi170.stdout:3.7 53 0 0 0 0 434176 0 0 53 0 53 active+clean 2023-12-01T06:34:10.714945+0000 43'53 45:73 [4,3,0] 4 [4,3,0] 4 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.492 INFO:tasks.workunit.client.0.smithi170.stdout:1.5 73 0 0 0 0 598016 0 0 73 0 73 active+clean+scrubbing 2023-12-01T06:42:36.705437+0000 43'73 45:103 [4,2,5] 4 [4,2,5] 4 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 scrubbing for 0s 0 0 2023-12-01T06:42:41.492 INFO:tasks.workunit.client.0.smithi170.stdout:2.6 64 0 0 0 0 524288 0 0 64 0 64 active+clean 2023-12-01T06:34:08.616014+0000 43'64 45:86 [1,0,4] 1 [1,0,4] 1 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.492 INFO:tasks.workunit.client.0.smithi170.stdout:3.6 64 0 0 0 0 524288 0 0 64 0 64 active+clean 2023-12-01T06:34:10.999966+0000 43'64 45:83 [0,1,4] 0 [0,1,4] 0 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.492 INFO:tasks.workunit.client.0.smithi170.stdout:1.4 51 0 0 0 0 417792 0 0 51 0 51 active+clean 2023-12-01T06:34:07.048410+0000 43'51 45:76 [3,0,1] 3 [3,0,1] 3 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.493 INFO:tasks.workunit.client.0.smithi170.stdout:2.7 53 0 0 0 0 434176 0 0 53 0 53 active+clean 2023-12-01T06:34:08.650762+0000 43'53 45:75 [3,0,2] 3 [3,0,2] 3 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.493 INFO:tasks.workunit.client.0.smithi170.stdout:3.1 67 0 0 0 0 548864 0 0 67 0 67 active+clean 2023-12-01T06:34:10.563124+0000 43'67 45:88 [4,3,5] 4 [4,3,5] 4 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.493 INFO:tasks.workunit.client.0.smithi170.stdout:1.3 54 0 0 0 0 442368 0 0 54 0 54 active+clean 2023-12-01T06:34:06.991139+0000 43'54 45:81 [4,1,2] 4 [4,1,2] 4 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.493 INFO:tasks.workunit.client.0.smithi170.stdout:2.0 72 0 0 0 0 589824 0 0 72 0 72 active+clean 2023-12-01T06:34:08.652076+0000 43'72 45:95 [3,5,1] 3 [3,5,1] 3 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.493 INFO:tasks.workunit.client.0.smithi170.stdout:3.0 72 0 0 0 0 589824 0 0 72 0 72 active+clean 2023-12-01T06:34:10.854449+0000 43'72 45:92 [1,2,4] 1 [1,2,4] 1 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.493 INFO:tasks.workunit.client.0.smithi170.stdout:1.2 55 0 0 0 0 450560 0 0 55 0 55 active+clean 2023-12-01T06:34:07.047458+0000 43'55 45:82 [0,4,1] 0 [0,4,1] 0 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.493 INFO:tasks.workunit.client.0.smithi170.stdout:2.1 67 0 0 0 0 548864 0 0 67 0 67 active+clean+scrubbing 2023-12-01T06:42:38.293167+0000 43'67 45:91 [2,5,0] 2 [2,5,0] 2 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 scrubbing for 0s 0 0 2023-12-01T06:42:41.493 INFO:tasks.workunit.client.0.smithi170.stdout:3.e 69 0 0 0 0 565248 0 0 69 0 69 active+clean 2023-12-01T06:34:10.999747+0000 43'69 45:87 [2,4,1] 2 [2,4,1] 2 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.493 INFO:tasks.workunit.client.0.smithi170.stdout:2.f 62 0 0 0 0 507904 0 0 62 0 62 active+clean 2023-12-01T06:34:08.509209+0000 43'62 45:85 [0,5,4] 0 [0,5,4] 0 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.493 INFO:tasks.workunit.client.0.smithi170.stdout:1.c 60 0 0 0 0 491520 0 0 60 0 60 active+clean 2023-12-01T06:34:07.050262+0000 43'60 45:84 [3,0,4] 3 [3,0,4] 3 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.493 INFO:tasks.workunit.client.0.smithi170.stdout:3.3 54 0 0 0 0 442368 0 0 54 0 54 active+clean 2023-12-01T06:34:10.714920+0000 43'54 44:74 [4,0,5] 4 [4,0,5] 4 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 periodic scrub scheduled @ 2023-12-02T07:27:01.574922+0000 0 0 2023-12-01T06:42:41.493 INFO:tasks.workunit.client.0.smithi170.stdout:1.1 67 0 0 0 0 548864 0 0 67 0 67 active+clean 2023-12-01T06:34:07.048236+0000 43'67 45:90 [5,0,4] 5 [5,0,4] 5 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.493 INFO:tasks.workunit.client.0.smithi170.stdout:2.2 55 0 0 0 0 450560 0 0 55 0 55 active+clean 2023-12-01T06:34:08.649704+0000 43'55 45:76 [5,1,0] 5 [5,1,0] 5 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.493 INFO:tasks.workunit.client.0.smithi170.stdout:3.2 55 0 0 0 0 450560 0 0 55 0 55 active+clean 2023-12-01T06:34:10.884368+0000 43'55 45:75 [3,5,0] 3 [3,5,0] 3 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.494 INFO:tasks.workunit.client.0.smithi170.stdout:1.0 72 0 0 0 0 589824 0 0 72 0 72 active+clean 2023-12-01T06:34:06.949811+0000 43'72 45:98 [1,0,2] 1 [1,0,2] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.494 INFO:tasks.workunit.client.0.smithi170.stdout:2.3 54 0 0 0 0 442368 0 0 54 0 54 active+clean 2023-12-01T06:34:08.651725+0000 43'54 45:75 [5,4,1] 5 [5,4,1] 5 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.494 INFO:tasks.workunit.client.0.smithi170.stdout:3.f 62 0 0 0 0 507904 0 0 62 0 62 active+clean 2023-12-01T06:34:10.999133+0000 43'62 43:80 [0,1,4] 0 [0,1,4] 0 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 periodic scrub scheduled @ 2023-12-02T08:16:03.451443+0000 0 0 2023-12-01T06:42:41.494 INFO:tasks.workunit.client.0.smithi170.stdout:2.e 69 0 0 0 0 565248 0 0 69 0 69 active+clean 2023-12-01T06:34:08.648989+0000 43'69 43:88 [2,3,5] 2 [2,3,5] 2 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 periodic scrub scheduled @ 2023-12-02T17:55:48.560107+0000 0 0 2023-12-01T06:42:41.494 INFO:tasks.workunit.client.0.smithi170.stdout:1.d 67 0 0 0 0 548864 0 0 67 0 67 active+clean 2023-12-01T06:34:06.994921+0000 43'67 43:88 [2,4,1] 2 [2,4,1] 2 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 periodic scrub scheduled @ 2023-12-02T08:55:33.837077+0000 0 0 2023-12-01T06:42:41.494 INFO:tasks.workunit.client.0.smithi170.stdout:3.c 60 0 0 0 0 491520 0 0 60 0 60 active+clean 2023-12-01T06:34:10.562236+0000 43'60 45:78 [5,3,1] 5 [5,3,1] 5 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.494 INFO:tasks.workunit.client.0.smithi170.stdout:2.d 67 0 0 0 0 548864 0 0 67 0 67 active+clean 2023-12-01T06:34:08.613058+0000 43'67 45:87 [5,1,3] 5 [5,1,3] 5 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.494 INFO:tasks.workunit.client.0.smithi170.stdout:1.e 69 0 0 0 0 565248 0 0 69 0 69 active+clean 2023-12-01T06:34:06.950499+0000 43'69 45:94 [1,4,2] 1 [1,4,2] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.494 INFO:tasks.workunit.client.0.smithi170.stdout:3.d 67 0 0 0 0 548864 0 0 67 0 67 active+clean 2023-12-01T06:34:10.883539+0000 43'67 45:86 [3,1,0] 3 [3,1,0] 3 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.494 INFO:tasks.workunit.client.0.smithi170.stdout:1.f 62 0 0 0 0 507904 0 0 62 0 62 active+clean 2023-12-01T06:34:07.048928+0000 43'62 45:86 [3,0,4] 3 [3,0,4] 3 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 queued for scrub 0 0 2023-12-01T06:42:41.494 INFO:tasks.workunit.client.0.smithi170.stdout:2.c 60 0 0 0 0 491520 0 0 60 0 60 active+clean 2023-12-01T06:34:08.651633+0000 43'60 43:79 [2,0,5] 2 [2,0,5] 2 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 periodic scrub scheduled @ 2023-12-02T13:47:15.084685+0000 0 0 2023-12-01T06:42:41.494 INFO:tasks.workunit.client.0.smithi170.stdout: 2023-12-01T06:42:41.494 INFO:tasks.workunit.client.0.smithi170.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2023-12-01T06:42:41.495 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:42:41.508 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:106: TEST_recover_unexpected: ceph pg dump pgs 2023-12-01T06:42:41.508 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:106: TEST_recover_unexpected: grep +scrubbing 2023-12-01T06:42:41.803 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:42:41.804 INFO:tasks.workunit.client.0.smithi170.stdout:3.9 61 0 0 0 0 499712 0 0 61 0 61 active+clean+scrubbing 2023-12-01T06:42:33.632021+0000 43'61 45:86 [4,2,5] 4 [4,2,5] 4 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 scrubbing for 4s 50 0 2023-12-01T06:42:41.804 INFO:tasks.workunit.client.0.smithi170.stdout:1.b 65 0 0 0 0 532480 0 0 65 0 65 active+clean+scrubbing 2023-12-01T06:42:38.485218+0000 43'65 45:94 [1,0,2] 1 [1,0,2] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 scrubbing for 2s 0 0 2023-12-01T06:42:41.804 INFO:tasks.workunit.client.0.smithi170.stdout:2.8 66 0 0 0 0 540672 0 0 66 0 66 active+clean+scrubbing 2023-12-01T06:42:33.880818+0000 43'66 45:94 [0,2,1] 0 [0,2,1] 0 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 scrubbing for 3s 50 0 2023-12-01T06:42:41.804 INFO:tasks.workunit.client.0.smithi170.stdout:1.9 61 0 0 0 0 499712 0 0 61 0 61 active+clean+scrubbing 2023-12-01T06:42:39.526485+0000 43'61 45:90 [1,4,5] 1 [1,4,5] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 scrubbing for 1s 0 0 2023-12-01T06:42:41.804 INFO:tasks.workunit.client.0.smithi170.stdout:1.8 66 0 0 0 0 540672 0 0 66 0 66 active+clean+scrubbing 2023-12-01T06:42:35.637257+0000 43'66 45:93 [5,4,3] 5 [5,4,3] 5 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 scrubbing for 4s 50 0 2023-12-01T06:42:41.804 INFO:tasks.workunit.client.0.smithi170.stdout:3.5 73 0 0 0 0 598016 0 0 73 0 73 active+clean+scrubbing 2023-12-01T06:42:37.571464+0000 43'73 45:95 [5,3,4] 5 [5,3,4] 5 0'0 2023-12-01T06:34:09.106331+0000 0'0 2023-12-01T06:34:09.106331+0000 0 0 scrubbing for 0s 0 0 2023-12-01T06:42:41.805 INFO:tasks.workunit.client.0.smithi170.stdout:1.7 53 0 0 0 0 434176 0 0 53 0 53 active+clean+scrubbing 2023-12-01T06:42:37.510163+0000 43'53 45:83 [1,3,0] 1 [1,3,0] 1 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 scrubbing for 3s 50 0 2023-12-01T06:42:41.805 INFO:tasks.workunit.client.0.smithi170.stdout:1.6 64 0 0 0 0 524288 0 0 64 0 64 active+clean+scrubbing 2023-12-01T06:42:39.493151+0000 43'64 45:92 [3,4,2] 3 [3,4,2] 3 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 scrubbing for 0s 0 0 2023-12-01T06:42:41.819 INFO:tasks.workunit.client.0.smithi170.stdout:1.5 73 0 0 0 0 598016 0 0 73 0 73 active+clean+scrubbing 2023-12-01T06:42:36.705437+0000 43'73 45:103 [4,2,5] 4 [4,2,5] 4 0'0 2023-12-01T06:34:05.269295+0000 0'0 2023-12-01T06:34:05.269295+0000 0 0 scrubbing for 0s 0 0 2023-12-01T06:42:41.819 INFO:tasks.workunit.client.0.smithi170.stdout:2.1 67 0 0 0 0 548864 0 0 67 0 67 active+clean+scrubbing 2023-12-01T06:42:38.293167+0000 43'67 45:91 [2,5,0] 2 [2,5,0] 2 0'0 2023-12-01T06:34:07.225310+0000 0'0 2023-12-01T06:34:07.225310+0000 0 0 scrubbing for 0s 0 0 2023-12-01T06:42:41.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:107: TEST_recover_unexpected: ok=true 2023-12-01T06:42:41.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:108: TEST_recover_unexpected: break 2023-12-01T06:42:41.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:112: TEST_recover_unexpected: test true = false 2023-12-01T06:42:41.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:117: TEST_recover_unexpected: local total=0 2023-12-01T06:42:41.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:118: TEST_recover_unexpected: local zerocount=0 2023-12-01T06:42:41.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:119: TEST_recover_unexpected: local maxzerocount=3 2023-12-01T06:42:41.820 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:42:41.821 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:42:41.822 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:42:41.823 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:42:41.824 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:41.825 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:41.825 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:41.825 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:41.825 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:41.825 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:41.825 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:41.825 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:41.826 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:41.826 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:41.921 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:41.921 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 1, 2023-12-01T06:42:41.921 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:41.921 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:41.921 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:41.921 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:41.935 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:41.935 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:41.935 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:41.935 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:41.935 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:41.935 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:41.935 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:41.935 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:41.936 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:41.936 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:42.058 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=1 2023-12-01T06:42:42.058 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 1 -gt 4 ']' 2023-12-01T06:42:42.058 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 1 2023-12-01T06:42:42.059 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=1 2023-12-01T06:42:42.059 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:42.061 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:42.061 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:42.061 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:42.061 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:42.061 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:42.061 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:42.061 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:42.061 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:42.062 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:42.180 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:42.180 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 3, 2023-12-01T06:42:42.180 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:42.180 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:42.180 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:42.180 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:42.195 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:42.195 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:42.195 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:42.195 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:42.195 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:42.195 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:42.196 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:42.196 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:42.196 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:42.196 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:42.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=3 2023-12-01T06:42:42.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 3 -gt 4 ']' 2023-12-01T06:42:42.312 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 1 + 3 2023-12-01T06:42:42.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=4 2023-12-01T06:42:42.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:42.314 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:42.314 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:42.314 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:42.315 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:42.315 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:42.315 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:42.315 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:42.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:42.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:42:42.423 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:42.423 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:42.423 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:42.423 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:42.423 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:42.423 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:42.438 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:42.438 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:42.438 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:42.438 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:42.439 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:42.439 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:42.439 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:42.439 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:42.439 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:42.440 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:42:42.561 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:42.561 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:42.562 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 4 + 2 2023-12-01T06:42:42.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=6 2023-12-01T06:42:42.564 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:42.565 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:42:42.565 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:42:42.565 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:42:42.565 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:42.566 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:42.566 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:42.566 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:42:42.566 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:42.566 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:42:42.676 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:42.677 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 3, 2023-12-01T06:42:42.677 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:42.677 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:42.677 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:42.677 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:42.690 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:42.691 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:42:42.691 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:42:42.691 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:42:42.692 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:42.692 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:42.692 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:42.692 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:42:42.693 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:42.693 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:42:42.810 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=3 2023-12-01T06:42:42.811 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 3 -gt 4 ']' 2023-12-01T06:42:42.811 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 6 + 3 2023-12-01T06:42:42.815 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=9 2023-12-01T06:42:42.815 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:42.816 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:42:42.816 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:42:42.816 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:42:42.816 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:42.816 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:42.816 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:42.816 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:42:42.816 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:42.816 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:42:42.923 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:42.923 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:42.923 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:42.923 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:42.923 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:42.923 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:42.938 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:42.938 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:42:42.939 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:42:42.939 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:42:42.939 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:42.939 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:42.939 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:42.939 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:42:42.939 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:42.939 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:42:43.056 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:43.056 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:43.057 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 9 + 2 2023-12-01T06:42:43.059 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=11 2023-12-01T06:42:43.059 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:43.059 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:42:43.059 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:42:43.059 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:42:43.060 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:43.060 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:43.060 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:43.061 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:42:43.061 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:43.061 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:42:43.167 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:43.167 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:43.167 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:43.167 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:43.168 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:43.168 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:43.182 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:43.184 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:42:43.184 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:42:43.184 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:42:43.184 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:43.184 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:43.185 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:43.185 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:42:43.185 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:43.185 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:42:43.309 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:43.310 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:43.310 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 11 + 2 2023-12-01T06:42:43.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=13 2023-12-01T06:42:43.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 13 = 0 ']' 2023-12-01T06:42:43.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 0 -gt 3 ']' 2023-12-01T06:42:43.312 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 0 + 13 2023-12-01T06:42:43.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=13 2023-12-01T06:42:43.314 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 13 -gt 0 ']' 2023-12-01T06:42:43.315 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:42:43.315 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:42:44.317 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:42:44.318 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:42:44.319 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:42:44.321 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:42:44.325 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:44.326 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:44.326 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:44.326 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:44.326 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:44.326 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:44.326 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:44.326 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:44.326 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:44.326 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:44.442 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:44.442 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:44.442 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:44.442 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:44.442 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:44.442 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:44.457 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:44.457 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:44.458 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:44.458 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:44.458 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:44.577 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:44.577 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:44.578 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 2 2023-12-01T06:42:44.579 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=2 2023-12-01T06:42:44.579 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:44.580 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:44.580 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:44.580 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:44.581 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:44.581 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:44.581 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:44.582 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:44.582 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:44.582 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:44.686 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:44.686 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:44.687 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:44.687 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:44.687 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:44.687 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:44.702 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:44.702 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:44.702 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:44.702 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:44.702 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:44.702 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:44.702 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:44.702 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:44.703 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:44.703 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:44.828 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:44.829 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:44.829 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 2 + 2 2023-12-01T06:42:44.831 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=4 2023-12-01T06:42:44.831 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:44.832 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:44.832 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:44.832 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:44.832 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:44.832 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:44.833 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:44.833 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:44.833 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:44.833 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:42:44.950 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:44.953 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:44.953 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:44.954 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:44.954 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:44.954 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:44.965 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:44.965 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:44.965 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:44.965 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:44.966 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:44.966 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:44.966 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:44.967 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:44.967 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:44.967 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:42:45.083 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:45.083 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:45.084 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 4 + 2 2023-12-01T06:42:45.085 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=6 2023-12-01T06:42:45.085 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:45.087 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:42:45.087 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:42:45.087 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:42:45.087 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:45.087 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:45.087 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:45.087 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:42:45.088 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:45.088 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:42:45.196 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:45.197 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 3, 2023-12-01T06:42:45.197 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:45.197 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:45.197 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:45.197 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:45.214 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:45.214 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:42:45.214 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:42:45.214 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:42:45.214 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:45.214 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:45.214 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:45.214 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:42:45.214 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:45.214 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:42:45.336 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=3 2023-12-01T06:42:45.336 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 3 -gt 4 ']' 2023-12-01T06:42:45.336 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 6 + 3 2023-12-01T06:42:45.337 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=9 2023-12-01T06:42:45.337 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:45.338 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:42:45.338 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:42:45.338 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:42:45.339 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:45.339 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:45.339 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:45.339 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:42:45.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:45.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:42:45.454 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:45.455 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:42:45.455 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:45.455 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:45.455 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:45.455 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:45.469 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:45.470 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:42:45.470 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:42:45.470 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:42:45.471 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:45.471 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:45.471 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:45.471 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:42:45.471 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:45.471 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:42:45.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:42:45.596 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:42:45.597 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 9 + 0 2023-12-01T06:42:45.599 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=9 2023-12-01T06:42:45.599 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:45.600 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:42:45.600 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:42:45.600 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:42:45.601 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:45.601 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:45.601 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:45.601 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:42:45.601 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:45.602 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:42:45.714 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:45.715 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:45.715 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:45.715 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:45.715 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:45.715 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:45.730 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:45.730 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:42:45.730 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:42:45.730 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:42:45.730 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:45.730 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:45.730 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:45.730 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:42:45.731 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:45.731 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:42:45.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:45.850 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:45.850 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 9 + 2 2023-12-01T06:42:45.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=11 2023-12-01T06:42:45.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 11 = 0 ']' 2023-12-01T06:42:45.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 0 -gt 3 ']' 2023-12-01T06:42:45.851 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 13 + 11 2023-12-01T06:42:45.852 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=24 2023-12-01T06:42:45.852 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 24 -gt 0 ']' 2023-12-01T06:42:45.854 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:42:45.855 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:42:46.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:42:46.857 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:42:46.859 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:42:46.860 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:42:46.861 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:46.862 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:46.863 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:46.863 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:46.863 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:46.863 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:46.864 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:46.864 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:46.864 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:46.864 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:46.981 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:46.981 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 3, 2023-12-01T06:42:46.982 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:46.982 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:46.982 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:46.982 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:46.996 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:46.996 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:46.996 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:46.996 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:46.997 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:46.997 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:46.997 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:46.997 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:46.998 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:46.998 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:47.116 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=3 2023-12-01T06:42:47.116 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 3 -gt 4 ']' 2023-12-01T06:42:47.116 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 3 2023-12-01T06:42:47.117 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=3 2023-12-01T06:42:47.117 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:47.118 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:47.118 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:47.119 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:47.119 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:47.119 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:47.120 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:47.120 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:47.120 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:47.120 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:47.230 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:47.231 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 1, 2023-12-01T06:42:47.231 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:47.231 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:47.231 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:47.231 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:47.246 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:47.246 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:47.246 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:47.246 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:47.246 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:47.247 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:47.247 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:47.247 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:47.247 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:47.247 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:47.369 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=1 2023-12-01T06:42:47.369 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 1 -gt 4 ']' 2023-12-01T06:42:47.370 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 3 + 1 2023-12-01T06:42:47.371 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=4 2023-12-01T06:42:47.371 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:47.372 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:47.372 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:47.372 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:47.373 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:47.373 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:47.373 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:47.373 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:47.374 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:47.374 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:42:47.475 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:47.476 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:47.476 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:47.476 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:47.476 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:47.476 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:47.489 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:47.489 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:47.489 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:47.489 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:47.490 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:47.490 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:47.490 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:47.491 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:47.491 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:47.491 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:42:47.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:47.604 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:47.605 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 4 + 2 2023-12-01T06:42:47.605 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=6 2023-12-01T06:42:47.606 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:47.607 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:42:47.608 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:42:47.608 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:42:47.608 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:47.608 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:47.608 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:47.608 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:42:47.609 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:47.609 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:42:47.717 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:47.717 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 1, 2023-12-01T06:42:47.717 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:47.717 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:47.717 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:47.717 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:47.732 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:47.732 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:42:47.732 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:42:47.732 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:42:47.733 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:47.733 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:47.733 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:47.733 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:42:47.733 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:47.733 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:42:47.848 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=1 2023-12-01T06:42:47.848 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 1 -gt 4 ']' 2023-12-01T06:42:47.848 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 6 + 1 2023-12-01T06:42:47.849 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=7 2023-12-01T06:42:47.850 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:47.850 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:42:47.850 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:42:47.850 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:42:47.851 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:47.851 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:47.851 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:47.851 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:42:47.851 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:47.852 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:42:47.960 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:47.960 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:42:47.960 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:47.960 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:47.961 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:47.961 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:47.975 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:47.975 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:42:47.975 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:42:47.975 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:42:47.976 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:47.976 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:47.976 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:47.976 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:42:47.976 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:47.976 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:42:48.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:42:48.092 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:42:48.093 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 7 + 0 2023-12-01T06:42:48.095 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=7 2023-12-01T06:42:48.095 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:48.095 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:42:48.095 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:42:48.096 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:42:48.096 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:48.096 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:48.097 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:48.097 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:42:48.097 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:48.097 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:42:48.204 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:48.204 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 3, 2023-12-01T06:42:48.204 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:48.204 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:48.204 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:48.204 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:48.219 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:48.219 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:42:48.219 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:42:48.219 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:42:48.220 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:48.220 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:48.220 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:48.220 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:42:48.220 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:48.220 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:42:48.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=3 2023-12-01T06:42:48.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 3 -gt 4 ']' 2023-12-01T06:42:48.339 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 7 + 3 2023-12-01T06:42:48.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=10 2023-12-01T06:42:48.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 10 = 0 ']' 2023-12-01T06:42:48.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 0 -gt 3 ']' 2023-12-01T06:42:48.341 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 24 + 10 2023-12-01T06:42:48.343 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=34 2023-12-01T06:42:48.343 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 34 -gt 0 ']' 2023-12-01T06:42:48.343 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:42:48.344 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:42:49.346 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:42:49.347 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:42:49.348 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:42:49.350 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:42:49.351 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:49.352 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:49.352 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:49.352 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:49.353 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:49.353 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:49.353 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:49.353 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:49.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:49.354 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:49.475 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:49.475 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 3, 2023-12-01T06:42:49.475 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:49.475 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:49.475 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:49.475 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:49.491 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:49.491 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:49.491 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:49.491 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:49.491 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:49.491 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:49.491 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:49.491 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:49.492 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:49.492 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:49.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=3 2023-12-01T06:42:49.610 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 3 -gt 4 ']' 2023-12-01T06:42:49.611 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 3 2023-12-01T06:42:49.611 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=3 2023-12-01T06:42:49.612 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:49.613 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:49.613 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:49.613 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:49.613 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:49.613 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:49.613 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:49.614 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:49.614 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:49.614 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:49.725 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:49.726 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 1, 2023-12-01T06:42:49.726 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:49.726 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:49.726 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:49.726 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:49.740 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:49.740 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:49.740 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:49.740 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:49.740 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:49.740 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:49.740 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:49.741 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:49.741 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:49.741 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:49.860 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=1 2023-12-01T06:42:49.861 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 1 -gt 4 ']' 2023-12-01T06:42:49.861 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 3 + 1 2023-12-01T06:42:49.863 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=4 2023-12-01T06:42:49.863 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:49.864 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:49.864 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:49.864 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:49.864 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:49.864 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:49.864 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:49.865 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:49.865 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:49.865 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:42:49.971 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:49.971 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 3, 2023-12-01T06:42:49.971 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:49.971 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:49.971 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:49.971 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:49.986 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:49.986 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:49.986 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:49.986 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:49.987 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:49.987 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:49.987 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:49.987 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:49.987 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:49.987 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:42:50.105 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=3 2023-12-01T06:42:50.105 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 3 -gt 4 ']' 2023-12-01T06:42:50.106 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 4 + 3 2023-12-01T06:42:50.107 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=7 2023-12-01T06:42:50.108 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:50.108 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:42:50.109 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:42:50.109 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:42:50.109 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:50.110 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:50.110 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:50.110 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:42:50.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:50.110 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:42:50.219 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:50.219 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 1, 2023-12-01T06:42:50.219 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:50.219 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:50.219 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:50.220 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:50.234 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:50.234 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:42:50.234 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:42:50.234 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:42:50.235 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:50.235 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:50.235 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:50.235 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:42:50.236 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:50.236 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:42:50.361 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=1 2023-12-01T06:42:50.362 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 1 -gt 4 ']' 2023-12-01T06:42:50.362 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 7 + 1 2023-12-01T06:42:50.363 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=8 2023-12-01T06:42:50.364 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:50.364 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:42:50.364 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:42:50.364 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:42:50.365 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:50.365 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:50.365 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:50.365 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:42:50.365 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:50.366 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:42:50.474 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:50.474 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:42:50.474 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:50.474 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:50.474 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:50.474 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:50.489 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:50.489 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:42:50.489 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:42:50.489 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:42:50.490 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:50.490 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:50.490 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:50.490 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:42:50.491 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:50.491 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:42:50.614 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:42:50.614 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:42:50.615 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 8 + 0 2023-12-01T06:42:50.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=8 2023-12-01T06:42:50.616 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:50.616 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:42:50.616 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:42:50.616 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:42:50.617 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:50.617 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:50.617 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:50.617 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:42:50.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:50.617 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:42:50.734 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:50.734 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:50.734 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:50.734 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:50.735 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:50.735 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:50.749 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:50.751 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:42:50.751 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:42:50.751 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:42:50.752 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:50.752 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:50.752 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:50.752 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:42:50.752 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:50.752 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:42:50.868 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:50.868 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:50.869 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 8 + 2 2023-12-01T06:42:50.870 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=10 2023-12-01T06:42:50.870 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 10 = 0 ']' 2023-12-01T06:42:50.871 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 0 -gt 3 ']' 2023-12-01T06:42:50.871 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 34 + 10 2023-12-01T06:42:50.872 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=44 2023-12-01T06:42:50.872 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 44 -gt 0 ']' 2023-12-01T06:42:50.872 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:42:50.874 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:42:51.876 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:42:51.876 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:42:51.878 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:42:51.879 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:42:51.880 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:51.882 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:51.882 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:51.882 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:51.882 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:51.882 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:51.882 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:51.883 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:51.883 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:51.883 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:51.998 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:51.998 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:51.998 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:51.998 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:51.998 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:51.998 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:52.012 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:52.012 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:52.012 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:52.012 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:52.013 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:52.013 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:52.013 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:52.013 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:52.013 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:52.014 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:52.138 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:52.138 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:52.139 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 2 2023-12-01T06:42:52.141 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=2 2023-12-01T06:42:52.141 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:52.142 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:52.142 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:52.142 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:52.143 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:52.143 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:52.143 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:52.143 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:52.143 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:52.143 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:52.252 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:52.252 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:52.252 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:52.252 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:52.253 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:52.253 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:52.267 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:52.268 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:52.268 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:52.268 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:52.268 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:52.268 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:52.269 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:52.269 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:52.269 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:52.269 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:52.388 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:52.388 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:52.388 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 2 + 2 2023-12-01T06:42:52.389 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=4 2023-12-01T06:42:52.390 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:52.390 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:52.390 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:52.391 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:52.391 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:52.391 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:52.391 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:52.391 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:52.392 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:52.392 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:42:52.501 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:52.501 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:52.501 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:52.501 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:52.502 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:52.502 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:52.516 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:52.516 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:52.516 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:52.516 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:52.516 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:52.516 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:52.517 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:52.517 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:52.517 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:52.517 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:42:52.642 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:52.642 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:52.643 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 4 + 2 2023-12-01T06:42:52.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=6 2023-12-01T06:42:52.644 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:52.645 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:42:52.645 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:42:52.645 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:42:52.646 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:52.646 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:52.646 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:52.646 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:42:52.647 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:52.647 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:42:52.757 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:52.758 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:52.758 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:52.758 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:52.758 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:52.758 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:52.773 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:52.773 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:42:52.773 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:42:52.773 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:42:52.774 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:52.774 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:52.774 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:52.774 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:42:52.774 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:52.774 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:42:52.892 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:52.892 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:52.893 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 6 + 2 2023-12-01T06:42:52.894 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=8 2023-12-01T06:42:52.894 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:52.895 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:42:52.895 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:42:52.895 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:42:52.896 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:52.896 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:52.896 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:52.896 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:42:52.896 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:52.896 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:42:53.007 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:53.007 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 3, 2023-12-01T06:42:53.007 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:53.007 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:53.007 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:53.007 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:53.022 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:53.022 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:42:53.022 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:42:53.023 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:42:53.023 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:53.024 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:53.024 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:53.024 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:42:53.024 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:53.024 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:42:53.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=3 2023-12-01T06:42:53.147 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 3 -gt 4 ']' 2023-12-01T06:42:53.147 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 8 + 3 2023-12-01T06:42:53.148 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=11 2023-12-01T06:42:53.148 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:53.149 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:42:53.149 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:42:53.149 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:42:53.150 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:53.150 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:53.150 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:53.150 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:42:53.150 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:53.150 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:42:53.269 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:53.269 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 1, 2023-12-01T06:42:53.269 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:53.269 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:53.269 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:53.269 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:53.283 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:42:53.284 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:42:53.284 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:42:53.284 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:53.284 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:53.285 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:53.285 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:53.285 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:42:53.285 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:53.285 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:42:53.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=1 2023-12-01T06:42:53.403 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 1 -gt 4 ']' 2023-12-01T06:42:53.404 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 11 + 1 2023-12-01T06:42:53.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=12 2023-12-01T06:42:53.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 12 = 0 ']' 2023-12-01T06:42:53.404 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 0 -gt 3 ']' 2023-12-01T06:42:53.405 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 44 + 12 2023-12-01T06:42:53.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=56 2023-12-01T06:42:53.407 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 56 -gt 0 ']' 2023-12-01T06:42:53.407 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:42:53.408 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:42:54.411 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:42:54.411 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:42:54.413 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:42:54.414 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:42:54.415 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:54.416 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:54.417 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:54.417 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:54.417 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:54.417 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:54.417 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:54.418 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:54.418 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:54.418 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:54.534 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:54.534 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:42:54.534 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:54.534 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:54.534 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:54.534 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:54.549 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:54.549 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:54.549 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:54.549 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:54.550 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:54.550 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:54.550 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:54.550 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:54.550 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:54.550 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:54.668 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:42:54.668 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:42:54.670 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:42:54.671 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:42:54.671 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:54.671 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:54.672 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:54.672 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:54.672 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:54.672 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:54.672 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:54.673 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:54.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:54.673 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:54.792 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:54.792 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 3, 2023-12-01T06:42:54.792 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:54.792 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:54.792 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:54.792 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:54.808 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:54.808 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:54.808 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:54.808 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:54.809 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:54.809 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:54.809 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:54.809 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:54.809 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:54.809 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:54.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=3 2023-12-01T06:42:54.923 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 3 -gt 4 ']' 2023-12-01T06:42:54.924 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 3 2023-12-01T06:42:54.924 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=3 2023-12-01T06:42:54.924 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:54.925 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:54.925 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:54.925 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:54.926 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:54.926 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:54.926 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:54.926 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:54.926 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:54.927 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:42:55.036 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:55.036 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:42:55.036 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:55.036 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:55.036 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:55.036 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:55.051 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:55.051 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:55.051 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:55.051 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:55.051 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:55.051 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:55.051 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:55.051 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:55.052 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:55.052 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:42:55.173 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:42:55.174 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:42:55.174 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 3 + 0 2023-12-01T06:42:55.175 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=3 2023-12-01T06:42:55.175 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:55.176 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:42:55.176 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:42:55.176 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:42:55.177 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:55.177 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:55.177 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:55.177 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:42:55.178 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:55.178 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:42:55.297 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:55.297 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:55.297 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:55.298 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:55.298 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:55.298 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:55.313 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:55.313 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:42:55.313 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:42:55.314 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:42:55.314 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:55.314 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:55.314 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:55.314 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:42:55.314 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:55.315 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:42:55.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=3 2023-12-01T06:42:55.432 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 3 -gt 4 ']' 2023-12-01T06:42:55.432 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 3 + 3 2023-12-01T06:42:55.433 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=6 2023-12-01T06:42:55.434 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:55.435 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:42:55.435 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:42:55.435 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:42:55.436 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:55.437 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:55.437 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:55.437 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:42:55.437 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:55.437 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:42:55.541 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:55.541 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 3, 2023-12-01T06:42:55.541 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:55.541 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:55.541 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:55.541 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:55.556 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:55.556 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:42:55.556 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:42:55.556 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:42:55.557 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:55.557 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:55.557 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:55.557 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:42:55.557 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:55.557 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:42:55.675 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=3 2023-12-01T06:42:55.675 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 3 -gt 4 ']' 2023-12-01T06:42:55.676 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 6 + 3 2023-12-01T06:42:55.676 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=9 2023-12-01T06:42:55.676 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:55.677 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:42:55.677 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:42:55.677 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:42:55.678 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:55.678 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:55.678 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:55.678 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:42:55.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:55.679 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:42:55.786 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:55.786 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 1, 2023-12-01T06:42:55.786 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:55.786 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:55.786 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:55.786 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:55.801 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:55.801 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:42:55.801 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:42:55.801 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:42:55.802 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:55.802 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:55.802 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:55.802 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:42:55.802 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:55.802 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:42:55.928 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=1 2023-12-01T06:42:55.928 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 1 -gt 4 ']' 2023-12-01T06:42:55.929 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 9 + 1 2023-12-01T06:42:55.931 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=10 2023-12-01T06:42:55.931 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 10 = 0 ']' 2023-12-01T06:42:55.931 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 0 -gt 3 ']' 2023-12-01T06:42:55.932 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 56 + 10 2023-12-01T06:42:55.934 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=66 2023-12-01T06:42:55.934 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 66 -gt 0 ']' 2023-12-01T06:42:55.935 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:42:55.935 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:42:56.938 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:42:56.939 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:42:56.940 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:42:56.942 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:42:56.943 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:56.944 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:56.944 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:56.944 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:56.945 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:56.945 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:56.945 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:56.945 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:56.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:56.946 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:57.063 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:57.063 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:42:57.063 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:57.063 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:57.063 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:57.063 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:57.078 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:57.079 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:57.079 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:57.079 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:57.079 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:57.079 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:57.080 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:57.080 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:57.080 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:57.080 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:57.197 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:42:57.197 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:42:57.197 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:42:57.198 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:42:57.198 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:57.199 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:57.199 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:57.199 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:57.200 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:57.200 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:57.200 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:57.201 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:57.201 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:57.201 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:57.312 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:57.312 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 3, 2023-12-01T06:42:57.312 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:57.312 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:57.312 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:57.313 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:57.328 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:57.328 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:57.328 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:57.328 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:57.328 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:57.328 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:57.328 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:57.328 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:57.328 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:57.328 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:57.453 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=3 2023-12-01T06:42:57.453 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 3 -gt 4 ']' 2023-12-01T06:42:57.453 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 3 2023-12-01T06:42:57.454 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=3 2023-12-01T06:42:57.454 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:57.455 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:57.455 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:57.455 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:57.455 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:57.455 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:57.456 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:57.456 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:57.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:57.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:42:57.567 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:57.568 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:42:57.568 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:57.568 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:57.568 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:57.568 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:57.582 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:57.582 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:57.583 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:57.583 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:57.583 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:57.583 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:57.584 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:57.584 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:57.584 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:57.584 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:42:57.701 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:42:57.702 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:42:57.702 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 3 + 0 2023-12-01T06:42:57.706 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=3 2023-12-01T06:42:57.706 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:57.706 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:42:57.706 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:42:57.706 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:42:57.706 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:57.706 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:57.706 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:57.706 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:42:57.706 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:57.707 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:42:57.827 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:57.827 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 4, 2023-12-01T06:42:57.827 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:57.827 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:57.827 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:57.827 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:57.842 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:57.842 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:42:57.842 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:42:57.842 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:42:57.843 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:57.843 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:57.843 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:57.843 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:42:57.843 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:57.843 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:42:57.962 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=4 2023-12-01T06:42:57.962 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 4 -gt 4 ']' 2023-12-01T06:42:57.962 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 3 + 4 2023-12-01T06:42:57.963 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=7 2023-12-01T06:42:57.964 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:57.964 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:42:57.964 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:42:57.964 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:42:57.965 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:57.965 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:57.965 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:57.965 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:42:57.966 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:57.966 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:42:58.070 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:58.071 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:58.071 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:58.071 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:58.071 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:58.071 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:58.087 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:58.087 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:42:58.087 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:42:58.087 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:42:58.088 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:58.088 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:58.088 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:58.088 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:42:58.088 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:58.088 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:42:58.213 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:58.213 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:58.214 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 7 + 2 2023-12-01T06:42:58.215 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=9 2023-12-01T06:42:58.215 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:58.216 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:42:58.216 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:42:58.216 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:42:58.217 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:58.217 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:58.217 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:58.217 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:42:58.217 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:58.217 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:42:58.336 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:58.336 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:58.336 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:58.337 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:58.337 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:58.337 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:58.352 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:58.352 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:42:58.352 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:42:58.352 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:42:58.353 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:58.353 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:58.353 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:58.353 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:42:58.353 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:58.353 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:42:58.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:58.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:58.474 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 9 + 2 2023-12-01T06:42:58.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=11 2023-12-01T06:42:58.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 11 = 0 ']' 2023-12-01T06:42:58.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 0 -gt 3 ']' 2023-12-01T06:42:58.477 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 66 + 11 2023-12-01T06:42:58.478 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=77 2023-12-01T06:42:58.478 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 77 -gt 0 ']' 2023-12-01T06:42:58.480 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:42:58.481 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:42:59.483 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:42:59.484 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:42:59.486 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:42:59.487 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:42:59.488 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:59.489 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:59.489 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:59.489 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:59.490 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:59.490 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:59.490 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:59.490 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:59.491 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:59.491 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:59.607 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:59.607 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:42:59.607 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:59.607 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:59.607 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:59.608 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:59.623 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:59.623 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:42:59.623 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:42:59.623 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:42:59.623 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:59.623 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:59.623 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:59.623 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:42:59.624 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:59.624 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:42:59.740 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:42:59.741 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:42:59.741 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:42:59.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:42:59.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:59.745 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:59.745 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:59.745 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:59.745 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:59.745 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:59.746 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:59.746 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:59.746 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:59.746 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:59.849 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:42:59.849 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:42:59.849 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:42:59.849 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:42:59.849 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:42:59.849 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:42:59.865 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:42:59.865 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:42:59.865 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:42:59.865 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:42:59.865 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:59.865 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:59.865 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:59.866 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:42:59.866 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:59.866 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:42:59.983 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:42:59.983 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:42:59.985 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 2 2023-12-01T06:42:59.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=2 2023-12-01T06:42:59.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:42:59.987 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:42:59.987 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:42:59.987 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:42:59.988 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:42:59.988 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:42:59.988 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:42:59.988 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:42:59.988 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:42:59.988 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:00.093 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:00.093 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:00.093 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:00.093 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:00.093 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:00.093 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:00.107 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:00.108 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:00.108 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:00.108 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:00.109 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:00.109 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:00.109 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:00.109 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:00.110 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:00.110 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:00.227 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:00.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:00.228 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 2 + 0 2023-12-01T06:43:00.230 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=2 2023-12-01T06:43:00.230 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:00.231 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:00.231 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:00.231 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:00.232 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:00.232 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:00.232 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:00.232 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:00.232 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:00.233 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:00.338 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:00.338 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 4, 2023-12-01T06:43:00.338 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:00.338 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:00.338 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:00.338 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:00.353 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:00.354 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:00.354 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:00.354 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:00.354 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:00.355 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:00.355 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:00.355 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:00.355 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:00.355 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:00.471 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=3 2023-12-01T06:43:00.471 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 3 -gt 4 ']' 2023-12-01T06:43:00.472 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 2 + 3 2023-12-01T06:43:00.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=5 2023-12-01T06:43:00.474 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:00.475 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:00.475 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:00.475 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:00.476 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:00.476 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:00.476 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:00.476 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:00.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:00.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:00.581 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:00.582 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 1, 2023-12-01T06:43:00.582 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:00.582 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:00.582 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:00.582 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:00.596 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:00.596 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:00.596 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:00.596 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:00.597 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:00.597 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:00.597 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:00.597 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:00.598 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:00.598 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:00.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=1 2023-12-01T06:43:00.719 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 1 -gt 4 ']' 2023-12-01T06:43:00.719 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 5 + 1 2023-12-01T06:43:00.720 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=6 2023-12-01T06:43:00.720 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:00.721 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:00.722 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:00.722 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:00.722 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:00.722 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:00.723 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:00.723 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:00.723 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:00.723 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:00.833 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:00.834 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:43:00.834 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:00.834 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:00.834 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:00.834 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:00.850 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:00.850 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:00.850 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:00.850 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:00.850 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:00.850 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:00.850 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:00.850 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:00.851 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:00.851 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:00.975 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:43:00.975 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:43:00.975 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 6 + 2 2023-12-01T06:43:00.976 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=8 2023-12-01T06:43:00.976 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 8 = 0 ']' 2023-12-01T06:43:00.976 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 0 -gt 3 ']' 2023-12-01T06:43:00.977 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 77 + 8 2023-12-01T06:43:00.977 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=85 2023-12-01T06:43:00.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 85 -gt 0 ']' 2023-12-01T06:43:00.978 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:43:00.979 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:43:01.982 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:43:01.983 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:43:01.983 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:43:01.985 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:43:01.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:01.987 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:01.987 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:01.987 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:01.988 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:01.989 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:01.989 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:01.989 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:01.989 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:01.989 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:02.105 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:02.105 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:02.106 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:02.106 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:02.106 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:02.106 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:02.120 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:02.120 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:02.120 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:02.120 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:02.121 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:02.121 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:02.121 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:02.121 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:02.121 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:02.122 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:02.238 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:02.238 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:02.239 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:02.242 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:02.242 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:02.242 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:02.242 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:02.243 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:02.243 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:02.243 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:02.243 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:02.243 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:02.243 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:02.243 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:02.352 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:02.352 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 1, 2023-12-01T06:43:02.352 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:02.352 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:02.352 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:02.352 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:02.367 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:02.368 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:02.368 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:02.368 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:02.368 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:02.368 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:02.368 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:02.369 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:02.369 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:02.369 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:02.485 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=1 2023-12-01T06:43:02.485 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 1 -gt 4 ']' 2023-12-01T06:43:02.486 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 1 2023-12-01T06:43:02.487 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=1 2023-12-01T06:43:02.487 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:02.488 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:02.489 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:02.489 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:02.489 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:02.489 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:02.489 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:02.490 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:02.490 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:02.490 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:02.586 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:02.587 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:02.587 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:02.587 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:02.587 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:02.587 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:02.600 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:02.600 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:02.600 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:02.600 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:02.601 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:02.601 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:02.601 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:02.601 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:02.601 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:02.601 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:02.710 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:02.710 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:02.711 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 1 + 0 2023-12-01T06:43:02.712 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=1 2023-12-01T06:43:02.712 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:02.713 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:02.713 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:02.713 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:02.713 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:02.713 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:02.714 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:02.714 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:02.714 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:02.714 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:02.820 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:02.821 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 3, 2023-12-01T06:43:02.821 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:02.821 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:02.821 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:02.821 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:02.836 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:02.836 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:02.836 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:02.836 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:02.836 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:02.836 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:02.836 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:02.836 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:02.837 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:02.837 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:02.952 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=3 2023-12-01T06:43:02.952 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 3 -gt 4 ']' 2023-12-01T06:43:02.952 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 1 + 3 2023-12-01T06:43:02.953 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=4 2023-12-01T06:43:02.954 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:02.954 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:02.954 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:02.954 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:02.955 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:02.955 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:02.955 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:02.956 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:02.956 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:02.956 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:03.066 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:03.067 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 1, 2023-12-01T06:43:03.067 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:03.067 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:03.067 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:03.067 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:03.082 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:03.082 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:03.082 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:03.082 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:03.082 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:03.082 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:03.082 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:03.083 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:03.083 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:03.083 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:03.206 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=1 2023-12-01T06:43:03.207 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 1 -gt 4 ']' 2023-12-01T06:43:03.210 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 4 + 1 2023-12-01T06:43:03.210 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=5 2023-12-01T06:43:03.210 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:03.211 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:03.211 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:03.211 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:03.212 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:03.212 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:03.212 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:03.212 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:03.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:03.212 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:03.322 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:03.322 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 1, 2023-12-01T06:43:03.322 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:03.322 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:03.322 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:03.322 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:03.337 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:03.337 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:03.337 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:03.337 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:03.338 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:03.338 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:03.338 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:03.338 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:03.338 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:03.339 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:03.455 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=1 2023-12-01T06:43:03.456 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 1 -gt 4 ']' 2023-12-01T06:43:03.456 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 5 + 1 2023-12-01T06:43:03.457 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=6 2023-12-01T06:43:03.458 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 6 = 0 ']' 2023-12-01T06:43:03.458 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 0 -gt 3 ']' 2023-12-01T06:43:03.458 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 85 + 6 2023-12-01T06:43:03.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=91 2023-12-01T06:43:03.459 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 91 -gt 0 ']' 2023-12-01T06:43:03.459 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:43:03.460 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:43:04.462 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:43:04.463 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:43:04.465 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:43:04.467 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:43:04.468 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:04.469 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:04.469 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:04.469 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:04.470 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:04.470 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:04.470 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:04.470 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:04.470 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:04.471 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:04.588 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:04.588 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:04.588 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:04.588 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:04.588 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:04.588 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:04.602 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:04.603 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:04.603 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:04.603 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:04.604 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:04.604 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:04.604 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:04.604 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:04.604 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:04.604 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:04.729 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:04.730 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:04.730 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:04.731 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:04.731 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:04.732 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:04.732 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:04.732 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:04.733 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:04.733 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:04.733 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:04.733 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:04.733 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:04.733 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:04.844 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:04.844 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 1, 2023-12-01T06:43:04.845 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:04.845 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:04.845 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:04.845 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:04.860 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:04.860 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:04.860 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:04.860 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:04.861 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:04.861 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:04.861 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:04.861 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:04.861 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:04.861 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:04.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=1 2023-12-01T06:43:04.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 1 -gt 4 ']' 2023-12-01T06:43:04.979 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 1 2023-12-01T06:43:04.980 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=1 2023-12-01T06:43:04.980 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:04.982 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:04.982 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:04.982 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:04.982 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:04.982 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:04.982 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:04.982 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:04.983 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:04.983 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:05.091 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:05.092 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:05.092 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:05.092 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:05.092 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:05.092 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:05.107 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:05.107 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:05.107 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:05.107 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:05.107 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:05.107 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:05.107 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:05.107 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:05.108 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:05.108 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:05.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:05.225 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:05.225 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 1 + 0 2023-12-01T06:43:05.226 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=1 2023-12-01T06:43:05.226 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:05.227 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:05.227 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:05.227 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:05.228 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:05.228 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:05.228 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:05.228 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:05.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:05.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:05.340 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:05.340 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 4, 2023-12-01T06:43:05.340 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:05.340 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:05.340 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:05.340 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:05.355 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:05.356 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:05.356 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:05.356 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:05.356 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:05.356 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:05.356 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:05.356 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:05.357 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:05.357 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:05.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=4 2023-12-01T06:43:05.475 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 4 -gt 4 ']' 2023-12-01T06:43:05.475 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 1 + 4 2023-12-01T06:43:05.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=5 2023-12-01T06:43:05.476 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:05.477 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:05.477 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:05.477 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:05.477 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:05.478 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:05.478 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:05.478 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:05.478 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:05.479 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:05.583 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:05.583 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:05.583 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:05.583 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:05.584 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:05.584 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:05.598 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:05.599 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:05.599 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:05.599 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:05.599 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:05.600 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:05.600 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:05.600 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:05.600 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:05.600 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:05.718 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:05.718 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:05.718 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 5 + 0 2023-12-01T06:43:05.720 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=5 2023-12-01T06:43:05.720 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:05.721 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:05.721 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:05.721 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:05.722 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:05.722 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:05.722 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:05.722 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:05.723 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:05.723 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:05.827 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:05.827 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:05.827 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:05.827 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:05.827 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:05.827 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:05.843 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:05.843 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:05.843 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:05.843 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:05.844 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:05.844 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:05.844 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:05.845 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:05.845 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:05.845 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:05.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:05.961 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:05.962 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 5 + 0 2023-12-01T06:43:05.963 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=5 2023-12-01T06:43:05.964 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 5 = 0 ']' 2023-12-01T06:43:05.964 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 0 -gt 3 ']' 2023-12-01T06:43:05.965 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 91 + 5 2023-12-01T06:43:05.966 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=96 2023-12-01T06:43:05.966 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 96 -gt 0 ']' 2023-12-01T06:43:05.967 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:43:05.967 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:43:06.970 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:43:06.970 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:43:06.971 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:43:06.973 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:43:06.974 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:06.976 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:06.976 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:06.976 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:06.977 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:06.977 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:06.977 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:06.977 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:06.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:06.978 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:07.088 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:07.089 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:07.089 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:07.089 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:07.089 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:07.089 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:07.103 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:07.103 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:07.103 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:07.103 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:07.105 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:07.105 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:07.106 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:07.106 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:07.106 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:07.106 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:07.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:07.228 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:07.229 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:07.231 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:07.231 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:07.232 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:07.232 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:07.232 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:07.233 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:07.233 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:07.233 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:07.233 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:07.234 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:07.234 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:07.353 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:07.353 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 1, 2023-12-01T06:43:07.353 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:07.353 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:07.353 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:07.353 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:07.368 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:07.368 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:07.368 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:07.368 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:07.369 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:07.369 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:07.369 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:07.369 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:07.370 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:07.370 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:07.486 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=1 2023-12-01T06:43:07.486 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 1 -gt 4 ']' 2023-12-01T06:43:07.487 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 1 2023-12-01T06:43:07.488 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=1 2023-12-01T06:43:07.488 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:07.489 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:07.489 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:07.489 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:07.489 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:07.489 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:07.489 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:07.490 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:07.490 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:07.490 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:07.602 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:07.602 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:07.602 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:07.602 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:07.602 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:07.603 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:07.617 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:07.617 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:07.617 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:07.618 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:07.618 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:07.618 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:07.618 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:07.618 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:07.618 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:07.618 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:07.741 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:07.741 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:07.742 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 1 + 0 2023-12-01T06:43:07.742 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=1 2023-12-01T06:43:07.743 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:07.743 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:07.743 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:07.743 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:07.744 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:07.744 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:07.745 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:07.745 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:07.745 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:07.745 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:07.855 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:07.855 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 2, 2023-12-01T06:43:07.855 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:07.855 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:07.855 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:07.855 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:07.869 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:07.870 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:07.870 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:07.870 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:07.871 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:07.871 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:07.871 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:07.871 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:07.871 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:07.871 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:07.995 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=2 2023-12-01T06:43:07.995 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 2 -gt 4 ']' 2023-12-01T06:43:07.996 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 1 + 2 2023-12-01T06:43:07.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=3 2023-12-01T06:43:07.998 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:08.000 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:08.000 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:08.000 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:08.000 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:08.000 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:08.000 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:08.000 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:08.000 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:08.000 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:08.104 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:08.104 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:08.104 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:08.104 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:08.105 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:08.105 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:08.119 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:08.120 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:08.120 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:08.120 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:08.120 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:08.120 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:08.120 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:08.120 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:08.121 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:08.121 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:08.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:08.245 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:08.245 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 3 + 0 2023-12-01T06:43:08.247 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=3 2023-12-01T06:43:08.247 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:08.248 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:08.248 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:08.248 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:08.248 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:08.248 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:08.248 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:08.249 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:08.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:08.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:08.354 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:08.355 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:08.355 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:08.355 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:08.355 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:08.355 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:08.369 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:08.370 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:08.370 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:08.370 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:08.370 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:08.371 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:08.371 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:08.371 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:08.371 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:08.371 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:08.493 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:08.493 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:08.494 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 3 + 0 2023-12-01T06:43:08.495 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=3 2023-12-01T06:43:08.496 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 3 = 0 ']' 2023-12-01T06:43:08.496 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 0 -gt 3 ']' 2023-12-01T06:43:08.496 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 96 + 3 2023-12-01T06:43:08.497 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=99 2023-12-01T06:43:08.497 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 99 -gt 0 ']' 2023-12-01T06:43:08.498 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:43:08.498 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:43:09.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:43:09.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:43:09.503 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:43:09.504 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:43:09.505 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:09.506 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:09.506 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:09.506 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:09.507 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:09.507 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:09.507 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:09.507 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:09.507 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:09.508 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:09.623 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:09.623 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:09.623 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:09.623 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:09.623 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:09.623 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:09.638 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:09.638 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:09.638 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:09.638 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:09.639 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:09.639 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:09.639 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:09.639 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:09.639 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:09.639 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:09.764 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:09.764 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:09.765 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:09.766 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:09.766 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:09.767 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:09.767 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:09.767 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:09.768 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:09.768 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:09.768 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:09.768 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:09.768 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:09.769 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:09.873 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:09.873 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:09.873 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:09.873 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:09.873 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:09.873 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:09.889 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:09.889 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:09.890 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:09.890 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:09.890 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:09.890 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:09.890 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:09.890 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:09.890 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:09.890 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:10.015 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:10.015 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:10.016 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:10.017 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:10.017 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:10.019 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:10.019 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:10.019 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:10.019 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:10.019 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:10.019 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:10.020 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:10.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:10.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:10.129 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:10.129 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:10.129 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:10.129 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:10.129 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:10.129 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:10.144 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:10.144 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:10.144 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:10.144 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:10.145 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:10.145 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:10.145 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:10.145 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:10.146 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:10.146 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:10.262 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:10.263 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:10.263 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:10.264 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:10.264 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:10.264 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:10.265 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:10.265 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:10.265 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:10.265 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:10.266 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:10.266 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:10.266 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:10.266 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:10.378 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:10.379 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 1, 2023-12-01T06:43:10.379 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:10.379 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:10.379 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:10.379 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:10.393 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:10.393 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:10.394 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:10.394 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:10.394 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:10.394 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:10.394 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:10.395 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:10.395 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:10.395 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:10.510 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=1 2023-12-01T06:43:10.510 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 1 -gt 4 ']' 2023-12-01T06:43:10.511 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 1 2023-12-01T06:43:10.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=1 2023-12-01T06:43:10.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:10.513 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:10.513 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:10.513 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:10.515 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:10.515 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:10.515 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:10.515 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:10.515 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:10.515 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:10.619 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:10.619 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:10.619 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:10.619 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:10.619 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:10.619 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:10.631 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:10.632 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:10.632 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:10.632 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:10.633 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:10.633 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:10.633 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:10.633 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:10.634 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:10.634 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:10.748 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:10.749 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:10.749 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 1 + 0 2023-12-01T06:43:10.751 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=1 2023-12-01T06:43:10.751 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:10.751 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:10.752 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:10.752 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:10.752 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:10.752 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:10.753 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:10.753 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:10.753 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:10.753 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:10.850 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:10.850 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:10.850 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:10.850 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:10.850 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:10.850 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:10.863 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:10.863 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:10.864 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:10.864 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:10.864 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:10.864 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:10.864 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:10.864 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:10.864 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:10.864 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:10.981 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:10.982 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:10.982 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 1 + 0 2023-12-01T06:43:10.984 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=1 2023-12-01T06:43:10.984 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 1 = 0 ']' 2023-12-01T06:43:10.984 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 0 -gt 3 ']' 2023-12-01T06:43:10.984 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 99 + 1 2023-12-01T06:43:10.985 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=100 2023-12-01T06:43:10.985 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 100 -gt 0 ']' 2023-12-01T06:43:10.986 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:43:10.988 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:43:11.989 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:43:11.989 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:43:11.992 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:43:11.992 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:43:11.993 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:11.995 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:11.995 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:11.995 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:11.996 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:11.996 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:11.996 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:11.996 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:11.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:11.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:12.111 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:12.112 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:12.112 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:12.112 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:12.112 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:12.112 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:12.127 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:12.127 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:12.128 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:12.128 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:12.128 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:12.128 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:12.128 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:12.128 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:12.128 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:12.128 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:12.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:12.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:12.245 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:12.247 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:12.247 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:12.248 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:12.248 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:12.248 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:12.248 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:12.248 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:12.249 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:12.249 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:12.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:12.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:12.354 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:12.354 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:12.355 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:12.355 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:12.355 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:12.355 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:12.370 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:12.370 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:12.370 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:12.370 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:12.370 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:12.370 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:12.370 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:12.371 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:12.371 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:12.371 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:12.478 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:12.478 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:12.479 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:12.480 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:12.480 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:12.480 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:12.480 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:12.480 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:12.481 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:12.481 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:12.481 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:12.481 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:12.481 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:12.482 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:12.592 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:12.592 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:12.592 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:12.592 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:12.592 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:12.592 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:12.607 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:12.607 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:12.607 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:12.607 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:12.607 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:12.607 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:12.607 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:12.607 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:12.608 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:12.608 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:12.732 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:12.732 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:12.733 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:12.735 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:12.735 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:12.735 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:12.735 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:12.735 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:12.736 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:12.736 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:12.736 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:12.737 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:12.737 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:12.737 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:12.853 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:12.853 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:12.853 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:12.853 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:12.853 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:12.853 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:12.868 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:12.868 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:12.868 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:12.868 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:12.869 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:12.869 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:12.869 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:12.869 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:12.869 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:12.869 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:12.992 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:12.992 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:12.993 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:12.995 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:12.995 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:12.996 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:12.996 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:12.996 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:12.997 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:12.997 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:12.997 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:12.997 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:12.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:12.997 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:13.107 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:13.107 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:13.107 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:13.108 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:13.108 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:13.108 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:13.121 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:13.122 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:13.122 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:13.122 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:13.123 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:13.123 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:13.123 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:13.123 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:13.123 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:13.124 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:13.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:13.250 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:13.250 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:13.252 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:13.252 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:13.253 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:13.253 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:13.253 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:13.254 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:13.254 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:13.254 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:13.254 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:13.254 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:13.254 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:13.365 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:13.365 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:13.365 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:13.366 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:13.366 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:13.366 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:13.383 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:13.507 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:13.507 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:13.507 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:13.507 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:13.507 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:13.507 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:13.507 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:13.507 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:13.507 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:13.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:13.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:13.509 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:13.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:13.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 0 = 0 ']' 2023-12-01T06:43:13.509 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:134: TEST_recover_unexpected: expr 0 + 1 2023-12-01T06:43:13.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:134: TEST_recover_unexpected: zerocount=1 2023-12-01T06:43:13.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 1 -gt 3 ']' 2023-12-01T06:43:13.509 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 100 + 0 2023-12-01T06:43:13.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=100 2023-12-01T06:43:13.510 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 100 -gt 0 ']' 2023-12-01T06:43:13.510 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:43:13.510 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:43:14.507 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:43:14.508 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:43:14.510 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:43:14.511 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:43:14.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:14.513 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:14.513 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:14.513 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:14.514 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:14.514 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:14.514 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:14.514 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:14.515 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:14.515 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:14.633 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:14.633 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:14.633 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:14.633 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:14.633 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:14.634 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:14.648 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:14.649 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:14.649 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:14.649 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:14.650 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:14.650 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:14.650 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:14.650 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:14.651 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:14.651 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:14.769 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:14.769 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:14.770 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:14.771 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:14.771 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:14.772 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:14.772 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:14.772 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:14.773 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:14.773 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:14.773 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:14.773 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:14.773 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:14.773 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:14.886 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:14.886 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:14.886 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:14.886 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:14.886 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:14.886 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:14.900 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:14.901 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:14.901 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:14.901 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:14.901 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:14.901 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:14.901 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:14.902 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:14.902 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:14.902 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:15.026 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:15.026 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:15.026 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:15.027 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:15.028 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:15.028 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:15.028 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:15.028 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:15.029 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:15.029 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:15.029 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:15.030 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:15.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:15.030 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:15.139 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:15.140 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:15.140 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:15.140 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:15.140 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:15.140 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:15.153 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:15.154 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:15.154 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:15.154 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:15.155 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:15.155 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:15.155 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:15.155 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:15.156 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:15.156 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:15.272 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:15.273 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:15.273 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:15.275 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:15.275 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:15.275 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:15.275 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:15.276 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:15.276 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:15.276 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:15.276 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:15.277 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:15.277 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:15.277 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:15.386 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:15.387 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:15.387 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:15.387 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:15.387 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:15.387 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:15.401 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:15.401 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:15.401 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:15.401 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:15.402 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:15.402 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:15.402 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:15.402 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:15.403 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:15.403 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:15.528 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:15.528 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:15.529 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:15.531 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:15.531 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:15.531 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:15.532 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:15.532 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:15.532 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:15.532 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:15.532 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:15.532 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:15.533 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:15.533 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:15.636 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:15.637 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:15.637 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:15.637 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:15.637 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:15.637 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:15.650 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:15.651 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:15.651 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:15.651 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:15.651 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:15.651 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:15.651 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:15.651 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:15.651 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:15.652 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:15.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:15.758 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:15.759 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:15.760 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:15.760 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:15.761 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:15.761 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:15.761 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:15.762 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:15.762 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:15.762 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:15.762 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:15.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:15.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:15.857 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:15.857 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:15.858 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:15.858 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:15.858 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:15.858 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:15.871 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:15.871 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:15.871 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:15.871 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:15.871 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:15.872 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:15.872 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:15.872 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:15.872 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:15.872 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:15.977 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:15.977 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:15.978 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:15.979 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:15.979 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 0 = 0 ']' 2023-12-01T06:43:15.979 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:134: TEST_recover_unexpected: expr 1 + 1 2023-12-01T06:43:15.980 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:134: TEST_recover_unexpected: zerocount=2 2023-12-01T06:43:15.980 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 2 -gt 3 ']' 2023-12-01T06:43:15.981 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 100 + 0 2023-12-01T06:43:15.982 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=100 2023-12-01T06:43:15.982 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 100 -gt 0 ']' 2023-12-01T06:43:15.982 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:43:15.983 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:43:16.985 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:43:16.985 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:43:16.987 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:43:16.989 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:43:16.990 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:16.992 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:16.992 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:16.992 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:16.992 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:16.992 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:16.992 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:16.992 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:16.993 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:16.993 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:17.109 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:17.109 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:17.109 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:17.109 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:17.109 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:17.109 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:17.123 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:17.124 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:17.124 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:17.124 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:17.125 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:17.125 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:17.125 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:17.125 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:17.125 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:17.125 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:17.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:17.249 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:17.250 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:17.251 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:17.251 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:17.251 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:17.252 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:17.252 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:17.252 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:17.252 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:17.252 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:17.253 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:17.253 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:17.253 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:17.357 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:17.357 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:17.357 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:17.357 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:17.357 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:17.358 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:17.373 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:17.373 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:17.373 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:17.373 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:17.373 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:17.374 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:17.374 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:17.374 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:17.374 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:17.374 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:17.497 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:17.497 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:17.498 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:17.499 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:17.499 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:17.500 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:17.500 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:17.500 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:17.501 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:17.501 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:17.501 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:17.501 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:17.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:17.501 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:17.604 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:17.605 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:17.605 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:17.605 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:17.605 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:17.605 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:17.620 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:17.620 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:17.620 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:17.620 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:17.620 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:17.620 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:17.620 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:17.620 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:17.621 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:17.621 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:17.744 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:17.745 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:17.745 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:17.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:17.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:17.750 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:17.750 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:17.750 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:17.750 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:17.750 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:17.750 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:17.750 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:17.750 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:17.751 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:17.857 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:17.858 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:17.858 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:17.858 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:17.858 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:17.858 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:17.870 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:17.871 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:17.871 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:17.871 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:17.872 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:17.872 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:17.872 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:17.872 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:17.872 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:17.873 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:17.985 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:17.985 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:17.985 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:17.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:17.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:17.987 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:17.987 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:17.987 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:17.988 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:17.988 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:17.988 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:17.988 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:17.988 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:17.988 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:18.104 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:18.104 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:18.104 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:18.104 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:18.104 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:18.104 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:18.118 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:18.119 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:18.119 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:18.119 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:18.119 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:18.120 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:18.120 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:18.120 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:18.120 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:18.120 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:18.243 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:18.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:18.244 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:18.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:18.244 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:18.245 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:18.245 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:18.245 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:18.245 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:18.246 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:18.246 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:18.246 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:18.246 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:18.246 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:18.362 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:18.362 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:18.362 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:18.362 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:18.362 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:18.362 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:18.377 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:18.378 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:18.378 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:18.378 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:18.378 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:18.378 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:18.379 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:18.379 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:18.379 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:18.379 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:18.503 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:18.503 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:18.504 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:18.506 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:18.506 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 0 = 0 ']' 2023-12-01T06:43:18.506 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:134: TEST_recover_unexpected: expr 2 + 1 2023-12-01T06:43:18.507 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:134: TEST_recover_unexpected: zerocount=3 2023-12-01T06:43:18.507 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 3 -gt 3 ']' 2023-12-01T06:43:18.508 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: expr 100 + 0 2023-12-01T06:43:18.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:139: TEST_recover_unexpected: total=100 2023-12-01T06:43:18.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:140: TEST_recover_unexpected: '[' 100 -gt 0 ']' 2023-12-01T06:43:18.510 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: expr 3 - 2 2023-12-01T06:43:18.510 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:143: TEST_recover_unexpected: sleep 1 2023-12-01T06:43:19.514 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:120: TEST_recover_unexpected: true 2023-12-01T06:43:19.514 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:122: TEST_recover_unexpected: pass=0 2023-12-01T06:43:19.515 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: expr 6 - 1 2023-12-01T06:43:19.516 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: seq 0 5 2023-12-01T06:43:19.517 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:19.518 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:19.518 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:19.518 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:19.518 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:19.519 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:19.519 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:19.519 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:19.519 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:19.519 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:19.634 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:19.635 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:19.635 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:19.635 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:19.635 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:19.635 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:19.648 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:19.649 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.0 2023-12-01T06:43:19.649 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-12-01T06:43:19.649 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-12-01T06:43:19.650 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:19.650 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:19.650 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:19.650 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.0.asok 2023-12-01T06:43:19.651 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:19.651 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.0.asok dump_scrub_reservations 2023-12-01T06:43:19.775 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:19.775 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:19.775 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:19.776 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:19.776 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:19.779 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:19.779 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:19.779 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:19.780 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:19.780 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:19.780 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:19.780 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:19.780 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:19.781 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:19.885 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:19.885 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:19.885 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:19.885 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:19.885 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:19.885 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:19.901 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:19.901 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.1 2023-12-01T06:43:19.901 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-12-01T06:43:19.901 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-12-01T06:43:19.901 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:19.901 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:19.901 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:19.901 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.1.asok 2023-12-01T06:43:19.901 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:19.901 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.1.asok dump_scrub_reservations 2023-12-01T06:43:20.017 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:20.017 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:20.018 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:20.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:20.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:20.019 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:20.019 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:20.019 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:20.020 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:20.020 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:20.020 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:20.020 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:20.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:20.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:20.132 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:20.132 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:20.132 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:20.132 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:20.132 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:20.132 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:20.146 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:20.147 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.2 2023-12-01T06:43:20.147 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-12-01T06:43:20.147 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-12-01T06:43:20.147 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:20.147 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:20.148 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:20.148 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.2.asok 2023-12-01T06:43:20.148 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:20.148 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.2.asok dump_scrub_reservations 2023-12-01T06:43:20.265 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:20.266 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:20.266 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:20.267 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:20.267 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:20.269 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:20.269 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:20.269 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:20.270 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:20.270 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:20.270 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:20.270 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:20.270 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:20.270 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:20.375 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:20.375 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:20.375 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:20.375 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:20.375 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:20.391 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:20.391 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.3 2023-12-01T06:43:20.391 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2023-12-01T06:43:20.391 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2023-12-01T06:43:20.391 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:20.391 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:20.391 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:20.392 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.3.asok 2023-12-01T06:43:20.392 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:20.392 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.3.asok dump_scrub_reservations 2023-12-01T06:43:20.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:20.509 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:20.509 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:20.510 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:20.510 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:20.511 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:20.511 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:20.511 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:20.512 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:20.512 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:20.512 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:20.512 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:20.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:20.512 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:20.624 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:20.624 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:20.624 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:20.624 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:20.624 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:20.624 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:20.640 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:20.640 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.4 2023-12-01T06:43:20.640 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.4 2023-12-01T06:43:20.640 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.4 ']' 2023-12-01T06:43:20.640 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:20.640 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:20.640 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:20.641 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.4.asok 2023-12-01T06:43:20.641 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:20.641 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.4.asok dump_scrub_reservations 2023-12-01T06:43:20.759 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:20.759 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:20.760 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:20.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:20.762 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:123: TEST_recover_unexpected: for o in $(seq 0 $(expr $OSDS - 1)) 2023-12-01T06:43:20.762 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:20.763 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:20.763 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:20.763 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:20.763 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:20.763 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:20.764 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:20.764 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:20.764 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:125: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:20.875 INFO:tasks.workunit.client.0.smithi170.stdout:{ 2023-12-01T06:43:20.875 INFO:tasks.workunit.client.0.smithi170.stdout: "scrubs_local": 0, 2023-12-01T06:43:20.875 INFO:tasks.workunit.client.0.smithi170.stdout: "granted_reservations": 0, 2023-12-01T06:43:20.875 INFO:tasks.workunit.client.0.smithi170.stdout: "PGs being served": "{}", 2023-12-01T06:43:20.875 INFO:tasks.workunit.client.0.smithi170.stdout: "osd_max_scrubs": 4 2023-12-01T06:43:20.875 INFO:tasks.workunit.client.0.smithi170.stdout:} 2023-12-01T06:43:20.889 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: jq '.scrubs_local + .granted_reservations' 2023-12-01T06:43:20.890 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: get_asok_path osd.5 2023-12-01T06:43:20.890 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2023-12-01T06:43:20.890 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2023-12-01T06:43:20.891 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-12-01T06:43:20.891 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:20.891 INFO:tasks.workunit.client.0.smithi170.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:20.891 INFO:tasks.workunit.client.0.smithi170.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.72213/ceph-osd.5.asok 2023-12-01T06:43:20.892 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: CEPH_ARGS= 2023-12-01T06:43:20.892 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: ceph daemon /tmp/ceph-asok.72213/ceph-osd.5.asok dump_scrub_reservations 2023-12-01T06:43:21.016 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:126: TEST_recover_unexpected: scrubs=0 2023-12-01T06:43:21.016 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:127: TEST_recover_unexpected: '[' 0 -gt 4 ']' 2023-12-01T06:43:21.017 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: expr 0 + 0 2023-12-01T06:43:21.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:131: TEST_recover_unexpected: pass=0 2023-12-01T06:43:21.019 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:133: TEST_recover_unexpected: '[' 0 = 0 ']' 2023-12-01T06:43:21.019 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:134: TEST_recover_unexpected: expr 3 + 1 2023-12-01T06:43:21.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:134: TEST_recover_unexpected: zerocount=4 2023-12-01T06:43:21.020 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:136: TEST_recover_unexpected: '[' 4 -gt 3 ']' 2023-12-01T06:43:21.021 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:137: TEST_recover_unexpected: break 2023-12-01T06:43:21.021 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:150: TEST_recover_unexpected: seq 0 5 2023-12-01T06:43:21.025 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:150: TEST_recover_unexpected: for i in $(seq 0 5) 2023-12-01T06:43:21.026 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:152: TEST_recover_unexpected: ceph pg dump pgs 2023-12-01T06:43:21.026 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:152: TEST_recover_unexpected: grep +scrubbing 2023-12-01T06:43:21.323 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:43:21.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:156: TEST_recover_unexpected: sleep 3 2023-12-01T06:43:24.342 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:150: TEST_recover_unexpected: for i in $(seq 0 5) 2023-12-01T06:43:24.342 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:152: TEST_recover_unexpected: ceph pg dump pgs 2023-12-01T06:43:24.343 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:152: TEST_recover_unexpected: grep +scrubbing 2023-12-01T06:43:24.651 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:43:24.669 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:156: TEST_recover_unexpected: sleep 3 2023-12-01T06:43:27.670 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:150: TEST_recover_unexpected: for i in $(seq 0 5) 2023-12-01T06:43:27.671 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:152: TEST_recover_unexpected: ceph pg dump pgs 2023-12-01T06:43:27.671 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:152: TEST_recover_unexpected: grep +scrubbing 2023-12-01T06:43:27.983 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:43:28.001 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:156: TEST_recover_unexpected: sleep 3 2023-12-01T06:43:31.002 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:150: TEST_recover_unexpected: for i in $(seq 0 5) 2023-12-01T06:43:31.002 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:152: TEST_recover_unexpected: ceph pg dump pgs 2023-12-01T06:43:31.003 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:152: TEST_recover_unexpected: grep +scrubbing 2023-12-01T06:43:31.315 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:43:31.333 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:156: TEST_recover_unexpected: sleep 3 2023-12-01T06:43:34.339 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:150: TEST_recover_unexpected: for i in $(seq 0 5) 2023-12-01T06:43:34.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:152: TEST_recover_unexpected: ceph pg dump pgs 2023-12-01T06:43:34.340 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:152: TEST_recover_unexpected: grep +scrubbing 2023-12-01T06:43:34.638 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:43:34.654 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:156: TEST_recover_unexpected: sleep 3 2023-12-01T06:43:37.656 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:150: TEST_recover_unexpected: for i in $(seq 0 5) 2023-12-01T06:43:37.656 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:152: TEST_recover_unexpected: ceph pg dump pgs 2023-12-01T06:43:37.657 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:152: TEST_recover_unexpected: grep +scrubbing 2023-12-01T06:43:37.966 INFO:tasks.workunit.client.0.smithi170.stderr:dumped pgs 2023-12-01T06:43:37.984 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:156: TEST_recover_unexpected: sleep 3 2023-12-01T06:43:40.986 INFO:tasks.workunit.client.0.smithi170.stdout:100 total reservations seen 2023-12-01T06:43:40.986 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:159: TEST_recover_unexpected: echo 100 total reservations seen 2023-12-01T06:43:40.987 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:164: TEST_recover_unexpected: expr 16 '*' 3 '*' 3 2023-12-01T06:43:40.988 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:164: TEST_recover_unexpected: actual_reservations=144 2023-12-01T06:43:40.988 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:165: TEST_recover_unexpected: '[' 100 -lt 144 ']' 2023-12-01T06:43:40.989 INFO:tasks.workunit.client.0.smithi170.stdout:ERROR: Unexpectedly low amount of scrub reservations seen during test 2023-12-01T06:43:40.989 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:166: TEST_recover_unexpected: echo 'ERROR: Unexpectedly low amount of scrub reservations seen during test' 2023-12-01T06:43:40.990 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:167: TEST_recover_unexpected: return 1 2023-12-01T06:43:40.990 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:46: run: return 1 2023-12-01T06:43:40.990 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2316: main: code=1 2023-12-01T06:43:40.990 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2318: main: teardown td/osd-scrub-dump 1 2023-12-01T06:43:40.990 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-dump 2023-12-01T06:43:40.990 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=1 2023-12-01T06:43:40.990 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-dump KILL 2023-12-01T06:43:40.991 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-12-01T06:43:40.991 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-12-01T06:43:40.991 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-12-01T06:43:40.991 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-12-01T06:43:40.991 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-12-01T06:43:41.306 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-12-01T06:43:41.306 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-12-01T06:43:41.307 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-12-01T06:43:41.307 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-12-01T06:43:41.308 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-12-01T06:43:41.309 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-12-01T06:43:41.309 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-12-01T06:43:41.310 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:43:41.310 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-12-01T06:43:41.311 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-12-01T06:43:41.311 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T06:43:41.312 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-12-01T06:43:41.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 1 = 1 ']' 2023-12-01T06:43:41.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: '[' -n '' ']' 2023-12-01T06:43:41.313 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:194: teardown: mkdir -p /home/ubuntu/cephtest/archive/log 2023-12-01T06:43:41.315 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:195: teardown: mv td/osd-scrub-dump/mgr.x.log td/osd-scrub-dump/mon.a.log td/osd-scrub-dump/osd.0.log td/osd-scrub-dump/osd.1.log td/osd-scrub-dump/osd.2.log td/osd-scrub-dump/osd.3.log td/osd-scrub-dump/osd.4.log td/osd-scrub-dump/osd.5.log /home/ubuntu/cephtest/archive/log 2023-12-01T06:43:41.316 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-dump 2023-12-01T06:43:41.340 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-12-01T06:43:41.340 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-12-01T06:43:41.341 INFO:tasks.workunit.client.0.smithi170.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.72213 2023-12-01T06:43:41.341 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.72213 2023-12-01T06:43:41.341 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-12-01T06:43:41.342 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-12-01T06:43:41.342 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2319: main: return 1 2023-12-01T06:43:41.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T06:43:41.343 INFO:tasks.workunit:Stopping ['scrub'] on client.0... 2023-12-01T06:43:41.343 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-12-01T06:43:41.631 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_bbeb098e76b9019fafbe1c7bd55e48604c324c89/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_bbeb098e76b9019fafbe1c7bd55e48604c324c89/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_bbeb098e76b9019fafbe1c7bd55e48604c324c89/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test scrub/osd-scrub-dump.sh) on smithi170 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bbeb098e76b9019fafbe1c7bd55e48604c324c89 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh' 2023-12-01T06:43:41.705 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=49832dc6d10b4bb9a0e6970062b3523a Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_bbeb098e76b9019fafbe1c7bd55e48604c324c89/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_bbeb098e76b9019fafbe1c7bd55e48604c324c89/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_bbeb098e76b9019fafbe1c7bd55e48604c324c89/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test scrub/osd-scrub-dump.sh) on smithi170 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bbeb098e76b9019fafbe1c7bd55e48604c324c89 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh' 2023-12-01T06:43:41.707 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-01T06:43:41.717 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-01T06:43:41.718 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-01T06:43:41.758 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-01T06:43:41.767 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-01T06:43:41.767 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-01T06:43:41.790 INFO:teuthology.orchestra.run.smithi170.stdout: remote refid st t when poll reach delay offset jitter 2023-12-01T06:43:41.790 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================== 2023-12-01T06:43:41.790 INFO:teuthology.orchestra.run.smithi170.stdout:*hv01.front.sepi 44.4.53.2 2 u 51 64 377 0.090 -8.189 0.675 2023-12-01T06:43:41.790 INFO:teuthology.orchestra.run.smithi170.stdout:+hv02.front.sepi 63.231.80.2 3 u 48 64 377 0.074 -5.931 0.701 2023-12-01T06:43:41.791 INFO:teuthology.orchestra.run.smithi170.stdout:+hv03.front.sepi 74.6.168.72 3 u 48 64 377 0.086 -7.360 0.641 2023-12-01T06:43:41.791 INFO:teuthology.orchestra.run.smithi170.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2023-12-01T06:43:41.792 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-01T06:43:41.801 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-01T06:43:41.802 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-01T06:43:41.826 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-01T06:43:41.862 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-01T06:43:41.896 INFO:teuthology.task.internal:Duration was 1909.186799 seconds 2023-12-01T06:43:41.897 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-01T06:43:41.926 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-01T06:43:41.927 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-01T06:43:41.983 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-01T06:43:41.983 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2023-12-01T06:43:41.984 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-01T06:43:41.994 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-01T06:43:41.994 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-01T06:43:42.079 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-01T06:43:42.079 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-01T06:43:42.174 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-01T06:43:42.183 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-01T06:43:42.183 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-01T06:43:42.197 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-01T06:43:42.206 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-01T06:43:42.249 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2023-12-01T06:43:42.275 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-01T06:43:42.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T06:43:42.319 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-01T06:43:42.328 INFO:teuthology.task.internal:Transferring archived files... 2023-12-01T06:43:42.334 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rfriedma-2023-11-30_18:48:32-rados-wip-rf-repl-hp-distro-default-smithi/7472795/remote/smithi170 2023-12-01T06:43:42.335 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-01T06:44:04.735 INFO:teuthology.task.internal:Removing archive directory... 2023-12-01T06:44:04.736 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-01T06:44:04.911 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-01T06:44:04.924 INFO:teuthology.task.internal:Not uploading archives. 2023-12-01T06:44:04.924 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-01T06:44:04.952 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-01T06:44:04.952 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-01T06:44:04.959 INFO:teuthology.orchestra.run.smithi170.stdout: 655472 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Dec 1 06:44 /home/ubuntu/cephtest 2023-12-01T06:44:04.959 INFO:teuthology.orchestra.run.smithi170.stdout: 673114 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Dec 1 06:21 /home/ubuntu/cephtest/mnt.0 2023-12-01T06:44:04.959 INFO:teuthology.orchestra.run.smithi170.stdout: 673115 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Dec 1 06:32 /home/ubuntu/cephtest/mnt.0/client.0 2023-12-01T06:44:04.959 INFO:teuthology.orchestra.run.smithi170.stdout: 800537 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Dec 1 06:42 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-01T06:44:04.959 INFO:teuthology.orchestra.run.smithi170.stdout: 800555 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Dec 1 06:43 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td 2023-12-01T06:44:04.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T06:44:04.960 INFO:teuthology.orchestra.run.smithi170.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-12-01T06:44:04.961 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi170 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-12-01T06:44:04.965 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-01T06:44:04.985 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-01T06:44:05.020 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi170 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-12-01T06:44:05.020 INFO:teuthology.nuke:Checking targets against current locks 2023-12-01T06:44:05.042 DEBUG:teuthology.nuke:shortname: smithi170 2023-12-01T06:44:05.042 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-01T06:44:05.062 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rfriedma-2023-11-30_18:48:32-rados-wip-rf-repl-hp-distro-default-smithi/7472795', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-01 06:03:37.073203', 'locked_by': 'scheduled_rfriedma@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK4CcJFpY/k+yeFg5uhqfolGWCEzEOC4S+Tv3otpRlwbuKIRwzhZwd2eOZ/bYw3QV/zDCwmDuj2AAKiVitqtaOo='} 2023-12-01T06:44:05.113 INFO:teuthology.orchestra.console.smithi170:Power off 2023-12-01T06:44:05.114 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-01T06:44:05.139 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off 2023-12-01T06:44:05.139 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T06:44:05.164 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2023-12-01T06:44:09.165 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T06:44:09.275 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2023-12-01T06:44:13.279 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T06:44:13.390 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2023-12-01T06:44:17.394 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T06:44:17.506 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2023-12-01T06:44:17.607 INFO:teuthology.orchestra.console.smithi170:Power off completed 2023-12-01T06:44:17.712 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} duration: 1909.1867988109589 failure_reason: 'Command failed (workunit test scrub/osd-scrub-dump.sh) on smithi170 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bbeb098e76b9019fafbe1c7bd55e48604c324c89 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh''' flavor: default owner: scheduled_rfriedma@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=49832dc6d10b4bb9a0e6970062b3523a status: fail success: false 2023-12-01T06:44:17.713 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-01T06:44:17.784 INFO:teuthology.run:FAIL